diff --git a/.claude/skills/sdk-dotnet/SKILL.md b/.claude/skills/sdk-dotnet/SKILL.md index eb05d76f4a..3567d4a09a 100644 --- a/.claude/skills/sdk-dotnet/SKILL.md +++ b/.claude/skills/sdk-dotnet/SKILL.md @@ -164,7 +164,7 @@ Hand-written in `Socket.cs`, because OpenAPI stops at the upgrade. One `Socket` and closes the agents it made. Dropping work would hang up on whoever is talking. - `GetOrCreateAgentAsync` keeps one agent per channel while its session is live, and opens chats with the message's `AgentId`, as Go and Python do. -- `Host(agent, toolTimeout)` sends `host_tools` for `agent.Tools` under `agent.Name` after every `ready`. Hosted calls are +- `Host(client, toolTimeout)` sends `host_tools` for `client.Tools` with `every_agent: true` after every `ready`; `client.RunDispatchAsync()` hosts them alone. Hosted calls are tracked work like calls, and `hosting_refused` throws out of `RunAsync`. A connection that ends without a normal close is redialled with backoff (1s doubling to 30s), as in Go. This is the one socket that reconnects: dispatch frames belong to one connection. diff --git a/.claude/skills/sdk-rust/SKILL.md b/.claude/skills/sdk-rust/SKILL.md index b31565e8e5..c8def82be8 100644 --- a/.claude/skills/sdk-rust/SKILL.md +++ b/.claude/skills/sdk-rust/SKILL.md @@ -120,8 +120,9 @@ returned. `close()` is idempotent. (or there was none). `active` and `handles` go on the handshake. A message with a `session_id` is answered with `answer`, as the server acting for the writer. `load` and `ping` every 15s. `get_or_create_agent` keeps one session per channel and closes them all when `run` ends. -- `Dispatch::host(&agent, tool_timeout)` declares the `AgentRef`'s own `tools` under its name - (`host_tools`) after every `ready`, runs each `tool_call` in +- `Dispatch::host(&client, tool_timeout)` declares the client's `tools()` for every agent + (`host_tools` with `every_agent: true`) after every `ready`, and `Client::run_dispatch` hosts + them alone; it runs each `tool_call` in the same `JoinSet`, and ends `run` with `Error::Failed` on `hosting_refused`. Unlike Go, it does not reconnect. - `Knowledge::add_url` and `Router::transcribe`/`record` poll until the job settles. diff --git a/.claude/skills/sdk/SKILL.md b/.claude/skills/sdk/SKILL.md index d971340ae8..9bac4e05ae 100644 --- a/.claude/skills/sdk/SKILL.md +++ b/.claude/skills/sdk/SKILL.md @@ -907,6 +907,17 @@ A session's tokens moved to `getSessionTokens` (`GET /v1/agents/sessions/{id}/to A running session's caller gives it tools and takes them back (no socket change): `POST /v1/agents/sessions/{id}/tools` (`addSessionTools`, body `AddSessionToolsRequest` `{tools: SessionTool[]}`, at least one) and `DELETE /v1/agents/sessions/{id}/tools/{name}` (`removeSessionTool`), both client-accessible and 204. A tool named as one the caller already has is a 409: to change one, remove it, then add it again, so no call runs half under each declaration. A change waits for the turn in progress and every tool call in flight, one answered as still running included, so a call is only checked against the tools it was made with, the model is offered the change from the next turn, and once a removal answers no call of the tool can still arrive: a handler goes then, and must not wait for a change from inside a call. Only the kind of caller that opened the session may: a device its own session's, a backend a backend's (403 otherwise). A name the session offers from elsewhere (built in, a connector's, a worker's, another agent's) is a 409; withdrawing a name the caller has no tool by, or changing a session that is not running, a 404; a speech-to-speech model whose tools are fixed when it opens (Gemini Live) a 400; a speech-to-speech session that drops while the change waits a 503, to ask again. A native model's call made while the tools change, or by a reply that opened while they did, is refused rather than run, and so is one of a tool it is not offered. A watcher that names the tools it runs (`?tools=`) is also asked to run the ones its side added, and leaves a call it does not hold to one that does, as every SDK's socket already does. Go has `stream.Pipeline.AddTools(ctx, *tools.Registry)` and `RemoveTool(ctx, name)`, with `client.Session` wrappers; the added functions are the session's alone, not its agent's registry, and are forgotten when it is left. Swift has `AgentSession.add(tools:)` and `remove(toolNamed:)`, and `swift_assistant`'s seat selector brings `reserve_table` that way. Every SDK has them now, the added functions kept per session: JavaScript `Session.addTools`/`removeTool`, Python (`plugins/stream`) `Session.add_tools`/`remove_tool` and `Accelerated.add_tools`/`remove_tool`, .NET `Session.AddToolsAsync`/`RemoveToolAsync`, Ruby `Session#add_tools`/`remove_tool`, Rust `Session::add_tools`/`remove_tool`, PHP `Session::addTools`/`removeTool`, and Kotlin and Dart `AgentSession.addTools`/`removeTool`, whose generators list the two operations. Each puts a handler in place before the request for a name it does not have yet, never replaces one it has, takes it back only when the router refused, marks a name whose adding got no answer unsettled and withdraws it (a 404 being as good) before the name is added again, lets a name withdrawn from its agent-wide registry be added again with another function, and makes one change at a time. A removal whose answer is lost is unsettled as an add's is, in every SDK, Go and Swift included: the handler stays until the next remove (a 404 accepted) or add (which withdraws first) of the name settles it. A change given up on while its request is in flight is unsettled the same way; Rust marks the name before the request, since a dropped future runs nothing after it. Only a 4xx is a refusal (an add's handlers taken back, a removal changing nothing): a 5xx, a 2xx that cannot be read, or no answer leaves the name unsettled, since a gateway can answer 502 or 504 after the router applied the change. The SDK's own HTTP stack resends neither request (a client the app hands in, with retries of its own, is the app's), so a lost answer never comes back as a replay's 404 or 409: JavaScript sends the DELETE as not idempotent, Ruby turns off Net::HTTP's one resend of it, PHP sends it on an Amp client without `RetryRequests`, .NET sends it with an empty body, since its handler resends a bodyless request whose connection closed before any answer, and Kotlin sends both as a one-shot body OkHttp never replays; Go, Python, Rust, Swift and Dart's stacks resend neither. A fork of a running session is offered the tools the parent has now (`forkSpec` takes the live spec, which a change of tools updates; `TestAForkOfARunningSessionIsOfferedTheToolsItHasNow`), so a fork watched with handlers gets the session's current ones (opening minus withdrawn, plus added), copied with the session's unsettled names and asked for under the change lock: Go `Session.Fork`, JavaScript, Python, .NET, Ruby, Rust, PHP and Dart. Such a fork waits for a change in flight, so it is not awaited from inside one of the session's tools either. Swift's and Kotlin's `fork` return a session the app attaches with its own tools. Python, .NET, Ruby, Rust and PHP keep a copy of the added functions, since a caller's registry there could later hold another function under the name; Go's and JavaScript's refuse a second one, and Swift's, Kotlin's and Dart's tools are values. +Tool groups (breaking): `tools` on `AgentConfigRequest`, `AgentConfigPatch` and `SyncAgentRequest` is now a list of `ToolGroup` (`name`, `loading` of `ToolLoading` `full`/`summary`/`deferred`, `tools`, `output_retention`), and the mapping it was (`AgentTools`: `progressive`, `output_retention`) is refused with a 400. `AgentConnectorBinding` gains `loading` and `output_retention` the same way. A session under a config is offered only the functions a group names: one its caller declares that no group names is a 400 `tool_not_configured` on create and on `addSessionTools`, and a worker's that no group names is not offered. `summary` is what `progressive: true` was; `deferred` holds a tool back until the model finds it with the router's `search_tool` and loads it with `load_tool`, names no function may take. `OfferedTool` (`listSessionTools`) carries `loading` for a live session. Go reads `tools:` in agent.yaml as the list (`sdks/go/agents/folder.go`, `toolGroupList`), refuses the old mapping saying where its settings went, and sends `loading` and `output_retention` on a connector entry; its examples name their functions in a group. Other SDKs need to update: Python (`vision_agents`' folder loader and `plugins/stream`), JavaScript (`sdks/js/src/folder.ts`), .NET, Ruby, Rust and PHP read `tools:` as the group list, refuse the mapping and send the connector fields, and their examples and docs name every function they register in a group, or those functions are refused; Swift, Kotlin and Dart need nothing but the regenerated types, since configs are server side. + +Tool call state (no breaking change): the session socket sends `tool_call.started`, `tool_call.progress` and `tool_call.finished` for every call of the session, whoever runs it (the router, a worker, a connector, a device), each carrying the call's whole `ToolCallState` (`tool_call_id`, `tool`, `turn_id`, `status` of `ToolCallStatus` `in_progress`/`waiting`/`success`/`failed`/`cancelled`, `progress`, `started_at`, `duration_ms`, `approved_at`, `rejected_at`, `error`), so a client replaces what it holds rather than merging. `waiting` is a call held for a person's approval, with its question as `progress`; a declined call or one whose wait lapsed ends `cancelled` with `rejected_at`. `tool_held` and `tool_updated` are now listed frames too. The schema is in the socket's `x-server-frames`. A function an agent's tool groups name that nothing on the session runs is logged as a warning on create, and a call of a hosted function no worker holds is warned of, while the agent carries on. Go has `stream.Pipeline.On(kind, fn) (off func())`, `ToolCall(id)` and `ToolCalls()`, with `client.Session.On`/`ToolCall`, and `Event.ToolCall`; JavaScript `session.toolCalls` (a store of a `ReadonlyMap`), `session.toolCall(id)`, `session.on(kind, fn)` and `useToolCalls`/`useToolCall`/`useSessionEvent` from `@stream-io/stream-agents/react`; Swift `@Published toolCalls` and `on(_:perform:) -> AnyCancellable`; Kotlin `toolCalls: StateFlow`, `toolCall(id): Flow` and `on(kind, handler): Job`; Dart `toolCalls: LiveValue` and `on(kind, handler)`. Each keeps finished calls, and the client SDKs no longer run an app tool's call sent with `run: false`, nor ask again for one sent `approved: true`. Other SDKs need to update: Python (`plugins/stream`), .NET, Ruby, Rust and PHP need the typed state, the three kinds and an `on(kind, handler)` on their session. + +A tool call no longer says how it is going (no wire change): every SDK dropped its call's update and say (Go `tools.Call.Update`/`Say`, with `tools.Line` only `Hold`; JavaScript and Python `ToolCallContext.update`/`say`, with `ToolLine` only `hold`; .NET `UpdateAsync`/`SayAsync`; Ruby, Rust and PHP `update`/`say`). A session's own `say` stays. The router still takes `tool_update` (with `verbatim`) from SDKs already released, so it stays in the spec and on the socket. Swift, Kotlin and Dart never had them. Nothing is left to update. + +Tool stats per agent (no breaking change): `getAgentStats` (`GET /v1/stats/agents`, server side only) takes `agent`, a config id (unknown is a 404 `agent_config_not_found`), and answers `sessions` and `avg_turns` beside the peak. Each `ToolStats` adds `sessions`, `session_share` (0 to 1, out of the window's sessions), `avg_turns_before_use`, `loading`, `definition_tokens` and `summary_tokens`: what deciding a tool's loading weighs. Go has the regenerated `GetAgentStatsParams.Agent` and fields, and no hand-written call. JavaScript and Python (`plugins/stream`) are regenerated. Other SDKs need to update: .NET, Ruby, Rust and PHP regenerate their types, and any that wrap the stats call pass `agent`; Swift, Kotlin and Dart need nothing, since it is server side. + +Tools hosted for every agent (no spec change): a worker's `host_tools` frame takes `every_agent: true` in place of an `agent_id`, and the router offers those tools to any session whose config names them in a tool group (never to a session without a config); a tool of the same name hosted for the agent itself wins. A frame with neither, or both, is `hosting_refused`; `hosting` and `hosting_refused` echo `every_agent`. Every server SDK registers the app's tools once on its client and hosts them with `every_agent`, and agent handles no longer carry tools: Go `client.Tools()` and `client.RunDispatch(ctx)` (`stream.ToolHost`, which `*client.Client` satisfies, is what `Dispatch.Host` takes), Python `Client.tools` and `run_dispatch()`, JavaScript `client.tools` and `runDispatch(signal)`, .NET `Tools` and `RunDispatchAsync`, Ruby `tools` and `run_dispatch`, Rust `tools()` and `run_dispatch()`, PHP `$tools` and `runDispatch()`. `dispatch.host(client)` is for a worker that also answers calls or messages. A client derived for a user (`AsUser`, `as_user`, `SetUser`) shares the registry. Other SDKs need to update: JavaScript's folder loader (`sdks/js/src/folder.ts`) refuses a top-level `tools:` in agent.yaml, so `travel_support_js` cannot name `lookup_booking` in a group and its agent is not offered it until the loader reads tool groups. + +Agent stats have an SDK method (no spec change): Go's `client.Agent.Stats(ctx, from, to)` looks up the agent's config by name and reads `getAgentStats` for its id, failing without a request for a name nothing is stored under (`sdks/go/client/client.go`). Other SDKs need to update: Python (`plugins/stream`), JavaScript, .NET, Ruby, Rust and PHP need the same `stats(from, to)` on their server-side agent; Swift, Kotlin and Dart need nothing, since it is server side. Dart catches up with Swift and Kotlin (no spec change): `AgentSession.interrupt({requestId})` sends `request_id` on the `interrupt` frame only when given, `say(text, interrupt: true)` sends a bare `interrupt` before the `say`, and `StreamAgents.scheduledTasks` is the Swift/Kotlin `ScheduledTasks` (`list`, `get`, `update`, `pause`, `resume`, `delete`, `runs`), its seven operations on Dart's generator allowlist, `update` never sending `instructions`. Dart spells the wire's `in` as `after` (`ScheduledTaskNext.after`, `ScheduledTaskSchedule.after`), as Kotlin does, and refuses `ScheduledTaskState.unknown` in a query with an `ArgumentError` before asking, as Kotlin does. Dart's generator now reads a property the spec requires but marks `nullable` as optional and always writes it, null included (`ScheduledTask.last_run_at` and `next_run_at`, `AddSessionToolsRequest.tools`). Still not in Swift, Kotlin or Dart, to be added to the three together: `tool_timeout_ms` on fork, the `config_id`, `state`, `created_at` and `custom` session filters, regenerate/edit/branches (`replaces`, `listResponseSiblings`, `getResponseBranch`, `activateResponse`), a document on `responses.create` (`files`), and answering an approval from a conversation's history (`answerConversationToolApproval`). None needs `resumeSession`: their `attach` reads the session with `getSession`, and the router keeps the declared tools and `tool_timeout_ms` on the session row, so resume is only needed to change them. Go `Agent.Sync` stores the config first (unless the stored config has a sandbox, AI-3363 below) and then its skills with `config_id` set (AI-3342): the router refuses a skill with no config (`skillComplaint`, `acceleration/internal/api/configs.go:1502`), and skill names are unique per config, so the skills are listed with `config_id` to find the ones to edit. `agents.DefineSkills` takes the `configID`. Other SDKs: Python already does this (`plugins/stream/vision_agents/plugins/stream/config.py:304-305`); JavaScript (`sdks/js/src/agent.ts:559`), Ruby (`sdks/ruby/lib/getstream/vision_agents/agent.rb:369`), .NET (`sdks/dotnet/src/GetStream.VisionAgents/Agent.cs:509`) and PHP send skills inside the `syncAgent` body and are not affected. **Rust needs the same fix**: `sdks/rust/src/agent.rs:675-690` (`sync_config`) creates or updates skills with `Skill::request()` (`sdks/rust/src/harness.rs:52`, no `config_id`) before it stores the config; store the config first, then list skills with `config_id` and write each with the config's id. An agent's UI instructions are kept apart from its instructions, so a session can turn UI off: `AgentConfig`, `AgentConfigRequest`, `AgentConfigPatch` and `SyncAgentRequest` have `ui_instructions` (an additive migration), and `CreateSessionRequest` has `ui`, a boolean a device may send. The router tells the model `ui_instructions` after the instructions (a backend's own session instructions included) unless the session was opened with `ui: false`, which reopening or forking it keeps (`agent_sessions.no_ui`). A sync sends the directory whole, so a sync without `ui_instructions` removes them; a PUT that leaves them out keeps them, so a client that does not know the field does not take the UI away. Go's `Load` compiles agent.yaml's `ui:` into `Folder.UIInstructions` and `syncFolder` sends it as `ui_instructions`; the folder's hash takes it in only when there is UI, so a directory without `ui:` is not synced again, and one with it is synced once more, which moves the UI out of the stored instructions. A router older than the field takes the sync and drops it, so `syncFolder` does not trust a stamp covering UI that the stored config lacks, and syncs again, and the router writes a sync of the same hash that declares UI the config lacks rather than finding it unchanged. Deploy the router before releasing an SDK that sends the field. A folder synced by both SDKs flips back: a JavaScript sync writes the UI into `instructions` and removes `ui_instructions`, so a session with `ui: false` is told the UI again; sync a folder with one SDK until JavaScript sends `ui_instructions`. The shared fixtures are unchanged: they are what the model is told, the instructions and the UI joined by a blank line, as the router joins them. Every generated client is regenerated (Go, JavaScript types, Python `plugins/stream`, Swift). Other SDKs need to update: JavaScript (`sdks/js/src/folder.ts`) should send the compiled `ui:` as `ui_instructions` rather than append it to `instructions`, its hash taking it in the same way; Python, .NET, Ruby, Rust and PHP, when they read `ui:`, the same; Swift has `SessionOptions.ui`, which `swift_assistant`'s Settings switch sets, and JavaScript's `sessions.create` passes `ui` through with the regenerated types; Python, Kotlin, Dart and the other server SDKs should offer `ui` on their create-session options, so an app that shows no UI can say so. diff --git a/.gitignore b/.gitignore index cd5e988110..14f305bacd 100644 --- a/.gitignore +++ b/.gitignore @@ -102,6 +102,7 @@ plugins/*/example/uv.lock /acceleration/say /acceleration/transcribe /examples/voice_agents/swift_demo/backend/backend +/examples/voice_agents/swift_assistant/backend/backend /examples/voice_agents/swift_demo/configure/configure /performance_benchmark/out/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 0fb14af7ee..ffd080e1e3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,13 @@ ## Features +### A tool is registered once for the app, and each agent's agent.yaml enables it + +Every server SDK's client has a tool registry for the whole app, and one call runs it for every agent: Go `client.Tools().Add(...)` and `client.RunDispatch(ctx)`, Python `client.tools.register(...)` and `await client.run_dispatch()`, JavaScript `client.tools.register(...)` and `await client.runDispatch()`, .NET `client.Tools.Register(...)` and `await client.RunDispatchAsync()`, Ruby `client.tools.register(...)` and `client.run_dispatch`, Rust `client.tools().register(...)` and `client.run_dispatch().await`, PHP `$client->tools->register(...)` and `$client->runDispatch()`. A worker that also answers calls or messages hosts them with `dispatch.host(client)`. A session is offered one of those tools only when a tool group of its agent's `agent.yaml` names it, and a tool of the same name an agent hosts itself wins. A worker asks for this with `every_agent: true` on its `host_tools` frame; one naming neither an agent nor `every_agent` is still refused, so a forgotten agent id never offers tools to every agent. + +### Tool stats show how to load each tool, per agent + +`GET /v1/stats/agents` takes `agent`, an agent config id, and answers how many sessions took a turn and how many turns they took. Each tool now also has the sessions that called it and their share, the turns a session took before its first call, how it was loaded, and what its definition costs every turn, in full and as a summary: the numbers that decide between `full`, `summary` and `deferred`. A call's token cost and its tool's definition size now survive tool output retention, which used to shrink a redacted call's tokens to its tool name. Calls recorded before this have no definition size. ### The Go SDK connects a user's own accounts `client.Connections()` creates, authorizes, reads, lists and deletes connections: `Create(ctx, client.ConnectionRequest{Connector: "linear", UserID: "alice"})`, then `Authorize(ctx, id, client.AuthorizeOptions{ReturnURL, UserID})` for the launch URL, handoff token and expiry of the consent, and `Get(ctx, id, client.ConnectionOptions{UserID})`, `List(ctx, client.ConnectionFilter{UserID, Connector})`, `Delete(ctx, id, client.ConnectionOptions{UserID})`. Every call takes the user as `UserID` of its options struct and sends it as both the owner and `X-Stream-User-Id`. An empty user is the app's own connection. @@ -629,6 +636,13 @@ Every reply the agent abandons logs one INFO line, `the agent was interrupted`, ## Breaking Changes +### Tools are hosted from the client, not from an agent + +An agent handle no longer has tools to host: Go `client.Agent.Tools()`, Python `Agent.register` and `Agent.functions`, JavaScript `AgentHandle.tools`, Rust `AgentRef.tools` and PHP `AgentHandle::$tools` are gone. Register the tools on the client and run them with its `run_dispatch` (see above), and name each one in a tool group of the agents that use it. `Dispatch.host` takes the client in place of an agent in every SDK, and the hosting refusal no longer names an agent: JavaScript's `HostingRefusedError` and PHP's `HostingRefusedException` lose `agentId`, and PHP's `Dispatch::$hosting` is a boolean. A session's own tools stay: Go `SessionOptions.Tools`, Python `SessionOptions(tools=...)`, and the in-process `Agent`'s tools in JavaScript, .NET, Ruby and Go's `agents` package. + +### A tool call no longer has update or say + +A tool call's update and say are gone from every SDK: Go `tools.Call.Update`/`Say` (and `tools.Line` has only `Hold`), JavaScript `ToolCallContext.update`/`say` (and `ToolLine` has only `hold`), Python `ToolCallContext.update`/`say` (and `ToolLine` has only `hold`), .NET `UpdateAsync`/`SayAsync`, Ruby `update`/`say`, Rust `update`/`say` and PHP `update()`/`say()`. A session's own `say` stays. A tool says nothing while it runs: the agent tells a waiting caller the work is still going, and a tool's `display_title` is what its call shows. The router still takes `tool_update` from an older SDK, so a released app keeps working. ### `agents.DefineSkills` takes the id of the agent config the skills belong to `agents.DefineSkills(ctx, backend, configID, skills)` stores skills under that config and edits only that config's own skill of each name. Without a config the router refuses a skill, so the old form never worked against it (#2074). diff --git a/README.md b/README.md index 686be3fa26..31cc8b6df0 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,12 @@ The combination of routing, persistence and memory makes it easy to switch the u Stay in control of how costs scale. Start with the frontier models and run your own when it makes sense. +## Skill + +The skill enables you to build complex AI apps in hours: + +* Ask Claude/Cursor/Lovable to setup Stream, with getstream.io/SKILL.md + ## Quickstart The heavy lifting is done in Go. But you can use the agents in any environment: @@ -115,7 +121,7 @@ Full apps with chat, voice and tools, one per client SDK: ## Tool Calling -Any backend SDK can provide tools to the agent. Here's a Python based example: +Any backend SDK can provide tools to the agent. Tools are registered once for the app, and each agent's `agent.yaml` names the ones it uses in a tool group. Here's a Python based example: ```python import asyncio @@ -123,27 +129,20 @@ import asyncio from vision_agents.core.llm import ToolCallContext from vision_agents.plugins import stream -agent = stream.Client().agent("my-agent") +client = stream.Client() -@agent.register() +@client.tools.register() async def customer_case_lookup(tool_call_context: ToolCallContext) -> list: """Look up the user's open support cases.""" user = tool_call_context.user if not user.id: raise ValueError("unknown user") - await tool_call_context.update("Checking your open cases") # pretend this calls your CRM return await crm.open_cases(user.id) -async def main() -> None: - dispatch = stream.Dispatch() - dispatch.host(agent, tool_timeout=30) - await dispatch.run() - - -asyncio.run(main()) +asyncio.run(client.run_dispatch()) ``` diff --git a/acceleration/api/openapi.yaml b/acceleration/api/openapi.yaml index 6ed32e01ef..0ab8b445bd 100644 --- a/acceleration/api/openapi.yaml +++ b/acceleration/api/openapi.yaml @@ -620,7 +620,10 @@ components: type: string type: object tools: - $ref: '#/components/schemas/AgentTools' + description: The functions an SDK provides that the agent may offer, in named groups. Empty when there are none, and then the agent offers no function. + items: + $ref: '#/components/schemas/ToolGroup' + type: array tts: type: string ui_instructions: @@ -770,7 +773,11 @@ components: type: string type: object tools: - $ref: '#/components/schemas/AgentTools' + description: Replaces the tool groups whole; an empty list removes them all. + items: + $ref: '#/components/schemas/ToolGroup' + maxItems: 64 + type: array tts: type: string ui_instructions: @@ -922,8 +929,11 @@ components: description: 'Cost labels, carried onto every request a session using it makes. The team tag is also the team those sessions are counted under, its spend limits included, when whoever opens one names none: neither the caller''s token nor the session''s team field. It is 1 to 59 letters, digits, _ . and -, and a config naming another team tag is refused. A config tagged draft_of, naming the config it copies, is a test copy: it answers no message on a channel connection and subscribes to no event, which stay with the live config, and it may bind a channel connection another config binds.' type: object tools: - $ref: '#/components/schemas/AgentTools' - description: How the agent is offered its connector tools. Left out on an update, the stored settings stay. + description: 'The functions an SDK provides that the agent may offer, in named groups, each with how it is loaded. A function an SDK registers, declares on a session or hosts in a worker is offered only when a group names it: a session declaring one that none names is refused (400 tool_not_configured), and a worker''s is left out. The router''s own tools, connector tools and other agents are not named here. Omitted or null on an update, the groups stored stay; an empty list removes them all. The mapping this field once was, with progressive and output_retention, is refused: loading summary and output_retention on a group or a connector binding replaced them.' + items: + $ref: '#/components/schemas/ToolGroup' + maxItems: 64 + type: array tts: type: string ui_instructions: @@ -958,10 +968,19 @@ components: $ref: '#/components/schemas/ConnectorBindingEvent' maxItems: 32 type: array + loading: + $ref: '#/components/schemas/ToolLoading' + description: How the binding's tools are offered to the model. Omitted is full. name: description: 'The alias, unique within the config: a lowercase letter, then up to 62 lowercase letters, digits, - or _, never __ and not ending in _. The model is offered each tool as __, split back at the first __, so a __ inside the alias or a _ at its end would split it in the wrong place.' pattern: ^[a-z]([a-z0-9_-]{0,61}[a-z0-9-])?$ type: string + output_retention: + description: How long the agent's sessions keep what the binding's tools were given and gave back on the turns they record, as a tool group's output_retention. Omitted or empty, it is kept as long as the session. + example: 30d + pattern: ^(|none|[1-9][0-9]*[hd])$ + patternDescription: none, a number of hours such as 12h, or a number of days such as 30d + type: string policy: $ref: '#/components/schemas/ConnectorBindingPolicy' description: How the binding's tool calls behave around speech and interruptions. Omitted, a call is cancelled at the provider when the turn is interrupted, and the agent picks its own words while it runs. @@ -1251,6 +1270,11 @@ components: type: object AgentStats: properties: + avg_turns: + description: Turns a session took in the window, on average. A turn is a reply to the person, not a rewound one or a delegated task's. Null when no session took one. + format: double + nullable: true + type: number peak_at: description: When that peak was first reached. Null when no session was open in the window. format: date-time @@ -1260,6 +1284,10 @@ components: description: The most sessions open at the same time, each one an agent running. A session still open counts up to now. format: int64 type: integer + sessions: + description: 'Sessions that took a turn in the window: what a tool''s session_share is out of.' + format: int64 + type: integer tool_calls: $ref: '#/components/schemas/ToolStats' description: Every tool call in the window, as one. Its tool is empty. @@ -1271,22 +1299,11 @@ components: required: - peak_concurrent_sessions - peak_at + - sessions + - avg_turns - tool_calls - tools type: object - AgentTools: - description: How an agent is offered its tools, and how long what they were given and gave back is kept. - properties: - output_retention: - description: 'How long this agent''s sessions keep what any of its tools were given and gave back on the turns they record: a tool''s arguments, its result, a search''s query and the wording of a tool''s error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed. Left out on a create, or empty, it is kept as long as the session; left out on an update, the stored setting stays, and an empty string removes it. The app''s and the organization''s policies (tool_output_retention) can shorten it and never lengthen it, and the shortest applies. It applies to tool calls recorded after it is set.' - example: 30d - pattern: ^(|none|[1-9][0-9]*[hd])$ - patternDescription: none, a number of hours such as 12h, or a number of days such as 30d - type: string - progressive: - description: Offer each tool by the first line of its description, with its arguments' descriptions left out, and have the first call to a tool return its full description and input schema instead of running it. It saves context on an agent with many tools, at the cost of one more model turn for each tool a conversation uses. Off by default. Left out on an update, the stored setting stays. - type: boolean - type: object AgentUser: properties: include_phone_history: @@ -5044,7 +5061,7 @@ components: ErrorDetail: properties: code: - description: 'What went wrong, for a program to branch on. Every type has a code of its own name (invalid_request, unauthenticated, forbidden, payment_required, not_found, method_not_allowed, not_acceptable, conflict, gone, payload_too_large, unsupported_media_type, rate_limited, internal_error, unavailable, timeout) that a failure has when nothing names it better. The others are validation_failed, missing_customer, missing_organization, server_side_only, not_configured (this deployment does not offer the feature), modality_not_routed, unsynced_changes (a sync asked to check would write over somebody''s edits), name_taken (a 409: another agent config, router config or voice, or another skill of the same agent config, already has the name, so another name will do), channel_connection_taken (a 409: another live agent config binds the channel connection as fixed, and one agent answers a channel connection''s messages, so the message names that config), call_unanswered (a 409: the call placed for a phone session ended with nobody answering, and the message names how: busy, no_answer, failed or canceled. Opening the session is refused, or, when the status came while the agent was joining the call, the session is ended at once and kept as ended), no_oauth_client (a 400: a consent found no OAuth client for the connector, and the message names each way the connector lets the app give it one), invalid_config (a 400: an agent config a run names does not start a session), insufficient_credits (a 402: the organization''s credit is spent, so work that costs money is refused until credit is added; a model that costs nothing is still served), system_limit_reached (a 402: the organization''s spend hard limit for the day or the month is reached, whatever its credit, and the message names the interval, the amount and when it resets), spend_limit_reached (a 402: one of the organization''s own spend limits is spent, on the organization, an app, a key, a usage category or a team, and the message says which and when it resets, so retrying before then is refused again), user_limit_reached (a 429: a spend limit on the one end user the request is for is spent, which stops that user and no one else, and Retry-After says when its window ends), spend_check_unavailable (a 503: the router could not share what the organization is spending with the other routers for a while, so work that costs money is refused until it can; a model that costs nothing is still served, and Retry-After says to try again in a few seconds), budget_moved (a 400: a policy sent a budget, which spend limits replaced), limit_above_parent (a 400: a spend limit is above a limit of its parent in the same interval, and the message names the parent''s limit), limit_exists (a 409: the subject already has a limit at that level and interval, and the message names its id), no_limit (a 404: the subject has no spend limit to reset or grant past), use_team (a 400: a category starting team: was sent, where the team field names a team), invalid_expiry (a 400: a grant''s expires_at is not in the future, or is more than 366 days away), too_many_grants (a 409: the organization has 20 live grants, as many as it may), tier_limit_reached (the organization''s tier limits refuse the work, and limit says which: a 403 for a call or a transfer, to a country the tier does not reach, past its minutes or recipients of the day, or past its calls at once; a 400 for an agent config whose sandbox is bigger or more open than the organization''s sandbox limits allow. The message names the tier and the limit), account_suspended (a 403: the organization is suspended, and every request is refused, one for a model that costs nothing included, until it is restored; the message says to contact support), concurrency_limited (a 429: the app has as many runs in flight as it may, and Retry-After says when to try again), model_failed (a 503: the model failed or gave no answer), prompt_injection (a 400: the prompt injection screen refused the input of a run or a tool result it read, which retrying an unchanged input does not fix), image_unavailable (a 400: an image a run was handed could not be fetched or is not a JPEG, PNG, WebP or GIF, which retrying the same image does not fix), document_too_large (a 413: a file sent with a response has more pages or text than one may, alone or with the files the conversation already holds, and the message names the limit), document_unreadable (a 400: a file sent with a response could not be read, such as one with a password, one that is damaged or one with no text in it, which retrying the same file does not fix), invalid_usage_categories (a 400: the usage categories the request names, in the token''s usage_categories and team claims or in X-Stream-Usage-Categories, are more than 10, one is not 1 to 64 characters of A-Z, a-z, 0-9, _ . : -, more than one names a team, or the team is over 59 characters), memory_unavailable (a 503: the memory store could not delete what was asked, so nothing that waits on it was deleted either, and the same request later can succeed), billing_not_enabled (a 404: this deployment bills nobody, or the caller''s app is in no organization it bills, so there is no account to read), provider_takes_no_key (a 400: a key was put for a provider whose work is never routed on a customer''s own key here, and the message names the ones that are), campaign_locked (a 409: the campaign''s state does not allow the change, such as a launched campaign''s sequence, or resuming one that is running), campaign_not_ready (a 400: a launch found the campaign missing what it needs to ring anybody, and the message says what), consent_required (a 400: a launch did not say the people on the campaign agreed to be called), branch_limit_reached (a 409: the conversation has as many edit and regenerate points as Stream Chat keeps for it, 100, so a regenerate or an edit is refused and the conversation is continued in a new chat; asking and reading are unaffected), branch_too_long (a 409: the branch an activation would bring back is more than 100 messages to send again, so it is continued in a new chat), branch_unavailable (a 410: the copies of a hidden branch were deleted, as a memory truncate deletes them, so it can be neither read nor brought back), switch_unfinished (a 409: a regenerate, an edit or an activation of the session failed after Stream Chat may have rewound, and finishing it still fails because Stream Chat does not answer, so the session''s questions, regenerates, edits and activations are refused meanwhile; try again: each one tries to finish it first, so it clears once Stream Chat answers), and _not_found for agent_config, call, campaign, channel_account, command, connection, grant, knowledge_document, knowledge_url, limit, provider_key, router_config, scheduled_task, session, simulation, simulation_run, skill and voice. A scheduled task also has invalid_schedule (a schedule that does not read: at and in both or neither, a date, time or duration that does not parse, or until without repeat), schedule_in_past, schedule_too_far (a one-shot, or an in, more than a year ahead), invalid_timezone, timezone_unknown (no zone was given and neither the user nor the config has one), invalid_repeat, interval_too_short (a run moved to less than 15 minutes from now and earlier than the run it replaces) and too_many_tasks (a 409: the user is at their active-task limit). More may be added, so a client should expect one it does not know.' + description: 'What went wrong, for a program to branch on. Every type has a code of its own name (invalid_request, unauthenticated, forbidden, payment_required, not_found, method_not_allowed, not_acceptable, conflict, gone, payload_too_large, unsupported_media_type, rate_limited, internal_error, unavailable, timeout) that a failure has when nothing names it better. The others are validation_failed, missing_customer, missing_organization, server_side_only, not_configured (this deployment does not offer the feature), modality_not_routed, unsynced_changes (a sync asked to check would write over somebody''s edits), name_taken (a 409: another agent config, router config or voice, or another skill of the same agent config, already has the name, so another name will do), channel_connection_taken (a 409: another live agent config binds the channel connection as fixed, and one agent answers a channel connection''s messages, so the message names that config), call_unanswered (a 409: the call placed for a phone session ended with nobody answering, and the message names how: busy, no_answer, failed or canceled. Opening the session is refused, or, when the status came while the agent was joining the call, the session is ended at once and kept as ended), no_oauth_client (a 400: a consent found no OAuth client for the connector, and the message names each way the connector lets the app give it one), invalid_config (a 400: an agent config a run names does not start a session), tool_not_configured (a 400: a session under an agent config was declared or given a function that none of the config''s tool groups names, and the message names it), insufficient_credits (a 402: the organization''s credit is spent, so work that costs money is refused until credit is added; a model that costs nothing is still served), system_limit_reached (a 402: the organization''s spend hard limit for the day or the month is reached, whatever its credit, and the message names the interval, the amount and when it resets), spend_limit_reached (a 402: one of the organization''s own spend limits is spent, on the organization, an app, a key, a usage category or a team, and the message says which and when it resets, so retrying before then is refused again), user_limit_reached (a 429: a spend limit on the one end user the request is for is spent, which stops that user and no one else, and Retry-After says when its window ends), spend_check_unavailable (a 503: the router could not share what the organization is spending with the other routers for a while, so work that costs money is refused until it can; a model that costs nothing is still served, and Retry-After says to try again in a few seconds), budget_moved (a 400: a policy sent a budget, which spend limits replaced), limit_above_parent (a 400: a spend limit is above a limit of its parent in the same interval, and the message names the parent''s limit), limit_exists (a 409: the subject already has a limit at that level and interval, and the message names its id), no_limit (a 404: the subject has no spend limit to reset or grant past), use_team (a 400: a category starting team: was sent, where the team field names a team), invalid_expiry (a 400: a grant''s expires_at is not in the future, or is more than 366 days away), too_many_grants (a 409: the organization has 20 live grants, as many as it may), tier_limit_reached (the organization''s tier limits refuse the work, and limit says which: a 403 for a call or a transfer, to a country the tier does not reach, past its minutes or recipients of the day, or past its calls at once; a 400 for an agent config whose sandbox is bigger or more open than the organization''s sandbox limits allow. The message names the tier and the limit), account_suspended (a 403: the organization is suspended, and every request is refused, one for a model that costs nothing included, until it is restored; the message says to contact support), concurrency_limited (a 429: the app has as many runs in flight as it may, and Retry-After says when to try again), model_failed (a 503: the model failed or gave no answer), prompt_injection (a 400: the prompt injection screen refused the input of a run or a tool result it read, which retrying an unchanged input does not fix), image_unavailable (a 400: an image a run was handed could not be fetched or is not a JPEG, PNG, WebP or GIF, which retrying the same image does not fix), document_too_large (a 413: a file sent with a response has more pages or text than one may, alone or with the files the conversation already holds, and the message names the limit), document_unreadable (a 400: a file sent with a response could not be read, such as one with a password, one that is damaged or one with no text in it, which retrying the same file does not fix), invalid_usage_categories (a 400: the usage categories the request names, in the token''s usage_categories and team claims or in X-Stream-Usage-Categories, are more than 10, one is not 1 to 64 characters of A-Z, a-z, 0-9, _ . : -, more than one names a team, or the team is over 59 characters), memory_unavailable (a 503: the memory store could not delete what was asked, so nothing that waits on it was deleted either, and the same request later can succeed), billing_not_enabled (a 404: this deployment bills nobody, or the caller''s app is in no organization it bills, so there is no account to read), provider_takes_no_key (a 400: a key was put for a provider whose work is never routed on a customer''s own key here, and the message names the ones that are), campaign_locked (a 409: the campaign''s state does not allow the change, such as a launched campaign''s sequence, or resuming one that is running), campaign_not_ready (a 400: a launch found the campaign missing what it needs to ring anybody, and the message says what), consent_required (a 400: a launch did not say the people on the campaign agreed to be called), branch_limit_reached (a 409: the conversation has as many edit and regenerate points as Stream Chat keeps for it, 100, so a regenerate or an edit is refused and the conversation is continued in a new chat; asking and reading are unaffected), branch_too_long (a 409: the branch an activation would bring back is more than 100 messages to send again, so it is continued in a new chat), branch_unavailable (a 410: the copies of a hidden branch were deleted, as a memory truncate deletes them, so it can be neither read nor brought back), switch_unfinished (a 409: a regenerate, an edit or an activation of the session failed after Stream Chat may have rewound, and finishing it still fails because Stream Chat does not answer, so the session''s questions, regenerates, edits and activations are refused meanwhile; try again: each one tries to finish it first, so it clears once Stream Chat answers), and _not_found for agent_config, call, campaign, channel_account, command, connection, grant, knowledge_document, knowledge_url, limit, provider_key, router_config, scheduled_task, session, simulation, simulation_run, skill and voice. A scheduled task also has invalid_schedule (a schedule that does not read: at and in both or neither, a date, time or duration that does not parse, or until without repeat), schedule_in_past, schedule_too_far (a one-shot, or an in, more than a year ahead), invalid_timezone, timezone_unknown (no zone was given and neither the user nor the config has one), invalid_repeat, interval_too_short (a run moved to less than 15 minutes from now and earlier than the run it replaces) and too_many_tasks (a 409: the user is at their active-task limit). More may be added, so a client should expect one it does not know.' type: string doc_url: description: Where the code is explained. @@ -6812,6 +6829,9 @@ components: description: description: What the model is told the tool does. type: string + loading: + $ref: '#/components/schemas/ToolLoading' + description: 'How the tool is offered, as its tool group or connector binding says: full for the router''s own. A deferred tool is listed once the model has loaded it. Absent for a session that has ended.' name: description: How the model asks for it. type: string @@ -7203,7 +7223,7 @@ components: application: support type: object tool_output_retention: - description: 'How long a session keeps what its tools were given and gave back on the turns it records: a tool''s arguments, its result, a search''s query and the wording of a tool''s error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed, so a session''s items read in the same order and tool usage still counts the call. Left out, or empty, it is kept as long as the session. The shortest of the organization''s, the app''s and the agent config''s tools.output_retention applies, so an agent can shorten it and never lengthen it. It applies to tool calls recorded after it is set. Deleting a session deletes all of it at once.' + description: 'How long a session keeps what its tools were given and gave back on the turns it records: a tool''s arguments, its result, a search''s query and the wording of a tool''s error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed, so a session''s items read in the same order and tool usage still counts the call. Left out, or empty, it is kept as long as the session. The shortest of the organization''s, the app''s and the output_retention of the agent config''s tool group or connector binding the tool is in applies, so an agent can shorten it and never lengthen it. It applies to tool calls recorded after it is set. Deleting a session deletes all of it at once.' example: 30d pattern: ^(|none|[1-9][0-9]*[hd])$ patternDescription: none, a number of hours such as 12h, or a number of days such as 30d @@ -10594,8 +10614,11 @@ components: type: string type: object tools: - $ref: '#/components/schemas/AgentTools' - description: How connector tools are offered. A setting left out keeps what is stored. + description: 'agent.yaml''s tools: the functions an SDK provides that the agent may offer, in named groups, as AgentConfigRequest''s tools names them, with the same rules. Sent, they replace the stored groups whole, an empty list removing them. Left out, the stored ones stay.' + items: + $ref: '#/components/schemas/ToolGroup' + maxItems: 64 + type: array tts: type: string ui_instructions: @@ -10976,6 +10999,101 @@ components: - tool_call_id - type type: object + ToolCallFrameType: + description: tool_call.started is the call's first frame, tool_call.progress one while it runs or waits, and tool_call.finished its last. + enum: + - tool_call.started + - tool_call.progress + - tool_call.finished + type: string + ToolCallState: + description: One change to a tool call's state. Every watcher is sent tool_call.started once, tool_call.progress for each change while it runs or waits, and tool_call.finished once, carrying the call's whole state each time, so a client keeps the last one it got per tool_call_id. + properties: + approved_at: + description: When a person allowed the call. + format: date-time + type: string + duration_ms: + description: From started_at to this change; on tool_call.finished, how long the call took. + format: int64 + type: integer + error: + description: Why a failed call failed, or what a declined one shows, such as "Not allowed in time". + type: string + progress: + description: 'The last thing said about the call while it runs, such as "Searching for flights": a tool_update, the connector binding''s pre_speech, or the question it waits on.' + type: string + rejected_at: + description: When a person declined the call, or its wait for them passed. + format: date-time + type: string + started_at: + format: date-time + type: string + status: + $ref: '#/components/schemas/ToolCallStatus' + tool: + type: string + tool_call_id: + description: The call, as tool_call, tool_started and tool_ran name it. + type: string + turn_id: + type: string + type: + $ref: '#/components/schemas/ToolCallFrameType' + required: + - type + - tool_call_id + - tool + - turn_id + - status + - started_at + - duration_ms + type: object + ToolCallStatus: + description: in_progress while it runs, waiting while it is held for a person's approval, and success, failed or cancelled once it finished. A call declined, or whose approval nobody gave in time, is cancelled with rejected_at. + enum: + - in_progress + - waiting + - success + - failed + - cancelled + type: string + ToolGroup: + description: A named set of functions an SDK provides that the agent may offer, how they are loaded and how long what they were given and gave back is kept. A function an SDK registers, declares on a session or hosts in a worker is offered only when a group names it. + properties: + loading: + $ref: '#/components/schemas/ToolLoading' + name: + description: What the group is called, unique within the config. + pattern: ^[a-z][a-z0-9_-]{0,62}$ + patternDescription: a lowercase letter, then up to 62 lowercase letters, digits, - or _ + type: string + output_retention: + description: 'How long the agent''s sessions keep what the group''s tools were given and gave back on the turns they record: a tool''s arguments, its result and the wording of its error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed. Left out or empty, it is kept as long as the session. The app''s and the organization''s policies (tool_output_retention) can shorten it and never lengthen it. It applies to tool calls recorded after it is set.' + example: 30d + pattern: ^(|none|[1-9][0-9]*[hd])$ + patternDescription: none, a number of hours such as 12h, or a number of days such as 30d + type: string + tools: + description: The functions' names, as the SDK registers them. A name sits in one group at most. + items: + type: string + maxItems: 128 + minItems: 1 + type: array + required: + - name + - tools + type: object + ToolLoading: + default: full + description: 'How a tool is offered to the model. full sends its whole description and input schema from the start. summary sends the first line of its description and its arguments without their descriptions, and the first call returns the full description and schema instead of running it: one more model turn for each tool a conversation uses, for less context on every turn. deferred offers nothing until the model needs it: an agent with a deferred tool is offered search_tool, which finds deferred tools by what they do, and load_tool, which returns their full descriptions and offers them from the model''s next step on.' + enum: + - full + - summary + - deferred + type: string ToolResultCommand: properties: error: @@ -11007,26 +11125,60 @@ components: description: What a call put in front of the model, its arguments and its result, estimated at four characters a token. format: double type: number + avg_turns_before_use: + description: 'Turns a session took before its first call, on average: 0 is a call on the first turn. Null on the row over every tool.' + format: double + nullable: true + type: number calls: format: int64 type: integer + definition_tokens: + description: What the tool's whole definition, its name, description and schema, costs every turn that carries it, estimated at four characters a token. Null when no call recorded it. + format: int64 + nullable: true + type: integer failed: description: Calls that ended in an error. format: int64 type: integer + loading: + description: 'How the latest call had the tool offered: full, summary or deferred. Empty when no call recorded it, as calls made before it was recorded and built-in tools.' + type: string p99_ms: format: double nullable: true type: number + session_share: + description: Sessions that called it, out of the window's sessions, from 0 to 1. Null when no session took a turn. + format: double + minimum: 0 + nullable: true + type: number + sessions: + description: Sessions that called it at least once. + format: int64 + type: integer + summary_tokens: + description: What its definition costs offered as a summary. Null unless its loading is summary. + format: int64 + nullable: true + type: integer tool: type: string required: - tool - calls - failed + - sessions + - session_share + - avg_turns_before_use - avg_tokens - avg_ms - p99_ms + - loading + - definition_tokens + - summary_tokens type: object TranscriptEntity: description: Something the recording named, for the providers that pick them out. @@ -18792,12 +18944,13 @@ paths: get: description: |- A WebSocket, which OpenAPI cannot describe past the upgrade. Frames are JSON objects carrying a `type` and the fields of that event. - The server sends what the conversation did: `joined`, `heard`, `responding`, `response_delta`, `reasoning_delta` (turn_id and text: the next piece of the model's thinking on a written reply, never spoken), `responded` (pending_work remains true while tools or delegated work are outstanding), `said` (a line the agent says as written rather than asking the model, such as its greeting, with the purpose it has), `speaking` (an utterance started being heard: turn_id, synthesis_id, the text the voice had been sent for it by then, at, when the call's track took its first audible frame, and delay_ms, for an utterance reported before the track got to it, how long after at it is expected to be heard, from the speech queued ahead of it, and message_start and message_end, where that text is in the reply's Chat message, whose text is not the voice's: directions a voice acts, digits it is told the count of and sentences held back are in one and not the other; a streaming voice's grows as it is sent more), `spoke` (an utterance finished being synthesised: turn_id, synthesis_id, characters (Unicode code points, as every character count on this socket is), audio_duration_ms, time_to_first_byte_ms, and message_start and message_end for all of it; it follows the utterance's `speaking`), `turn`, `decision`, `delegated`, `task_settled` (files lists what the work's code handed back, each a name, mime_type, url and size, uploaded to a persistent conversation's channel and attached to the reply), `task_cancelled`, `tool_call`, `tool_held`, `tool_approval_ignored`, `tool_updated`, `tool_ran`, `transferred`, `pressed`, `looked_up`, `backchannel`, `interrupted` (turn_id; with synthesis_id, heard_ms, message_start and message_end, the utterance being heard when it was cut, how many milliseconds of it had been, and where all it had been sent is in the reply's message, when any of it had), `overlap_decided`, `conversation_compacted`, `models_changed`, `error` and `left`. + The server sends what the conversation did: `joined`, `heard`, `responding`, `response_delta`, `reasoning_delta` (turn_id and text: the next piece of the model's thinking on a written reply, never spoken), `responded` (pending_work remains true while tools or delegated work are outstanding), `said` (a line the agent says as written rather than asking the model, such as its greeting, with the purpose it has), `speaking` (an utterance started being heard: turn_id, synthesis_id, the text the voice had been sent for it by then, at, when the call's track took its first audible frame, and delay_ms, for an utterance reported before the track got to it, how long after at it is expected to be heard, from the speech queued ahead of it, and message_start and message_end, where that text is in the reply's Chat message, whose text is not the voice's: directions a voice acts, digits it is told the count of and sentences held back are in one and not the other; a streaming voice's grows as it is sent more), `spoke` (an utterance finished being synthesised: turn_id, synthesis_id, characters (Unicode code points, as every character count on this socket is), audio_duration_ms, time_to_first_byte_ms, and message_start and message_end for all of it; it follows the utterance's `speaking`), `turn`, `decision`, `delegated`, `task_settled` (files lists what the work's code handed back, each a name, mime_type, url and size, uploaded to a persistent conversation's channel and attached to the reply), `task_cancelled`, `tool_call`, `tool_held`, `tool_approval_ignored`, `tool_started`, `tool_updated`, `tool_ran`, `tool_call.started`, `tool_call.progress`, `tool_call.finished`, `transferred`, `pressed`, `looked_up`, `backchannel`, `interrupted` (turn_id; with synthesis_id, heard_ms, message_start and message_end, the utterance being heard when it was cut, how many milliseconds of it had been, and where all it had been sent is in the reply's message, when any of it had), `overlap_decided`, `conversation_compacted`, `models_changed`, `error` and `left`. An `error` frame for a turn the prompt injection screen refused carries `code` `prompt_injection`, and the turn's `responded` has empty text: what streamed before the verdict is not the answer. message_start and message_end are offsets in Unicode code points into the text of the Chat message the reply is written into, the one whose turn_id names the turn, from message_start up to message_end. They are left out where the router does not know them. `connector_unavailable` names an optional connector binding the session opened without: name (its alias), connector_id and reason, one of no_selection, shared_session, caller_unverified, connection_unavailable, provider_mismatch, needs_reauthorization, credential_rejected (the provider rejected the token or key a bearer or api_key connection holds, or the connection reads a connector revision marked broken; only saving its credentials again fixes it, so no login is offered), not_connected, open_failed, tool_unavailable and selection_dropped (a fork's or a reopened chat's selection for an alias its config no longer declares). Every watcher is sent each one when it attaches. `connector_scope_required` says a connector tool call was refused because the caller's own connection lacks access the provider asked for (insufficient_scope or a claims challenge), and a step-up consent was begun for it: name (the binding's alias), connector_id, connection_id, scopes (what the provider asked for, empty for a claims challenge), authorization_id, launch_url, handoff_token and expires_at. A page of the session's return_url's origin, or the dashboard for a session created without one, opens launch_url in a popup and posts it handoff_token, as for a createAuthorization with that return_url: the launch page takes the handoff from that origin only, and the consent goes back to the session's return_url, or else to the dashboard. The old grant keeps working until the step-up succeeds, and the same call works afterwards in the same session. While that step-up is open, calls refused for the same access send no second event. Persistent text sessions also emit `conversation_updated` with conversation_id and a complete message snapshot: id, request_id, question_id, role, text, state, response_started_at, state_started_at, finished_at, duration_ms, saved, persistence_error, refused (true on a reply the prompt injection screen refused) and attachments. Each tool_calling attachment has tool_call_id, name, title, status, phase, summary, immutable started_at, execution_started_at, finished_at and duration_ms. A connector_authorization attachment asks the end user to connect a connector binding the reply needed with their own account, with name (the binding's alias), connector_id, connection_id, authorization_id, title, launch_url, handoff_token and expires_at: a page of the session's return_url's origin, or the dashboard for a session created without one, opens launch_url in a popup and posts it handoff_token, as for a createAuthorization with that return_url, and the consent goes back to the session's return_url, or else to the dashboard. Once the user finishes that login the message is sent again with status connected and no handoff_token, and the agent carries on by itself. The call that needed the login waits for it on its own ai_tool_call step, in parts: status awaiting_authorization, target_user_id the caller, and an authorization of type oauth with provider (the connector id), provider_name, title, message, url (the launch page), handoff_token and expires_at. A page of the return_url's origin opens url and posts it handoff_token; a native app opens url#handoff_token= in its web authentication sheet. Connected, the step is running again with authorization.status connected and no handoff_token, and the call runs in the same reply. A tool_approval naming its tool_call_id with allowed false, or the person refusing at the provider, declines the login (declined); a consent that ended otherwise without connecting, such as with another account, is failed, and one not finished by expires_at is expired. Each way the step is cancelled, authorization.status says how, and the call runs nothing. Activity states are thinking, queued, tools, writing, completed, failed and cancelled. tool_started includes tool_call_id, tool, turn_id and started_at, and pre_speech when the tool's connector binding sets one in its policy; tool_ran also includes tool_call_id, and files when the tool was run_code, as on task_settled. + `tool_call.started`, `tool_call.progress` and `tool_call.finished` (`ToolCallState`) are a call's state as a client shows it, whoever runs the call: tool_call_id, tool, turn_id, status (in_progress, waiting for a person's approval, success, failed or cancelled), progress (the last `tool_update`, the binding's pre_speech or the approval question), started_at, duration_ms, approved_at, rejected_at and error. Each carries the whole state: keep the last per tool_call_id. A respond command carrying request_id emits command_accepted with a nested command receipt (request_id, user_message_id, assistant_message_id, state, duplicate). Personal persistent text sessions require this ID. A retry with the same text and images returns the existing IDs without invoking the model again; reuse with different content emits an error. Its images are data URIs, at most 8, uploaded to the conversation's channel and shown on the person's message. On a session a dispatch worker opened for a message handed out from a channel, that message is already there: nothing is written or uploaded for it, user_message_id is empty, and a reused id is answered as a duplicate even with other text or images. After restart an interrupted command is reported, not rerun. An `interrupt` command carrying `request_id` stops that command and emits `command_stopped` with its terminal receipt. A stop arriving after its command finished replays that command's receipt and leaves the command running now alone; an unknown command is reported as an error. Without `request_id` the frame stops whichever reply is current, which is what a caller with no command to name means by it. A `decision` frame is one judgement the conversation made, carrying the same fields as a CallEvent. Together they are why the call went the way it did, and they are also written down, so a finished call replays them from `/v1/agents/calls/{id}/events`. @@ -18871,6 +19024,8 @@ paths: - $ref: '#/components/schemas/SessionRespondCommand' - $ref: '#/components/schemas/ToolResultCommand' - $ref: '#/components/schemas/ToolApprovalCommand' + x-server-frames: + - $ref: '#/components/schemas/ToolCallState' /v1/agents/sessions/{id}/fork: post: description: |- @@ -26559,7 +26714,8 @@ paths: /v1/stats/agents: get: description: |- - The most agents that were running at once, and which tools they called: how often, how much each call put in front of the model and how long it took. + The most agents that were running at once, and which tools they called: how often, in how many sessions and after how many turns, how much each call put in front of the model, what the tool's definition costs every turn that carries it and how long a call took. + With agent, the sessions, turns and tools are that agent config's alone; the peak is still every agent's. One figure for the whole window rather than a row per bucket, because neither a peak nor a percentile can be summed out of smaller ones. operationId: getAgentStats parameters: @@ -26581,6 +26737,13 @@ paths: description: End of the window, exclusive. format: date-time type: string + - description: 'The id of an agent config: only its sessions are counted. Omitted is every agent.' + explode: false + in: query + name: agent + schema: + description: 'The id of an agent config: only its sessions are counted. Omitted is every agent.' + type: string responses: "200": content: @@ -26605,6 +26768,8 @@ paths: $ref: '#/components/responses/PaymentRequired' "403": $ref: '#/components/responses/Forbidden' + "404": + $ref: '#/components/responses/NotFound' "500": $ref: '#/components/responses/InternalError' "503": diff --git a/acceleration/internal/agent/agent.go b/acceleration/internal/agent/agent.go index 4253726083..4d56daf4a9 100644 --- a/acceleration/internal/agent/agent.go +++ b/acceleration/internal/agent/agent.go @@ -301,8 +301,11 @@ type Agent struct { switching atomic.Bool // swapping serialises swaps. swapping sync.Mutex - // toolsMu guards options.Tools, which SetTools replaces while turns read it. + // toolsMu guards options.Tools, which SetTools replaces while turns read it, and + // loadedTools. toolsMu sync.RWMutex + // loadedTools are the deferred tools the model loaded with load_tool, in the order it did. + loadedTools []string llm *llmrouter.Session // replies is where every reply in flight is fanned in to, so the one goroutine that @@ -4708,6 +4711,7 @@ func turnOf(synthesisID string) string { // so the people already heard keep their labels when a session switches between voice and // writing. func (a *Agent) Continue(previous *Agent) { + a.offerLoaded(previous.loaded()) previous.mu.Lock() history := append([]llm.Message(nil), previous.history...) compactedAway := append([]llm.Message(nil), previous.compactedAway...) @@ -4734,6 +4738,7 @@ func (a *Agent) Continue(previous *Agent) { // The authors of turns read back from a thread channel are counted as speakers, in the order // they first wrote, and once two have written each of their turns goes under its label. func (a *Agent) RestoreHistory(history []llm.Message) { + a.offerLoaded(loadedIn(history)) a.mu.Lock() defer a.mu.Unlock() a.history = append([]llm.Message(nil), history...) diff --git a/acceleration/internal/agent/approval.go b/acceleration/internal/agent/approval.go index 23008d34c0..16db4632c5 100644 --- a/acceleration/internal/agent/approval.go +++ b/acceleration/internal/agent/approval.go @@ -16,6 +16,10 @@ import ( // question. It is offered only on a voice call with a tool that asks one. const confirmTool = "confirm_tool_call" +// ConfirmTool is the tool the model answers a call's approval question with. The call it +// confirms runs inside it, so its ToolRan is that call's outcome too. +const ConfirmTool = confirmTool + // approvalWait is how long a voice caller has to answer an approval question, from when it // finished playing. const approvalWait = 15 * time.Second @@ -167,6 +171,7 @@ func (a *Agent) askApproval(call llm.ToolCall, question string) []llm.ContentPar } a.mu.Unlock() a.logger.Info("a call waits for the caller's yes", "tool", call.Name, "tool_call", call.ID) + a.emitter.Send(ToolAwaitingApproval{ID: call.ID, TurnID: call.TurnID, Tool: call.Name, Question: question}) return llm.TextParts(fmt.Sprintf(answer, call.Name, question, call.ID, call.Name)) } diff --git a/acceleration/internal/agent/deferred_tools.go b/acceleration/internal/agent/deferred_tools.go new file mode 100644 index 0000000000..c9e9829184 --- /dev/null +++ b/acceleration/internal/agent/deferred_tools.go @@ -0,0 +1,118 @@ +package agent + +import ( + "encoding/json" + "fmt" + "slices" + + "github.com/GetStream/Vision-Agents/acceleration/internal/harness" + "github.com/GetStream/Vision-Agents/acceleration/internal/llm" +) + +// defers reports whether the model finds and loads some of its tools as it needs them +// (harness.Tools.Deferred). A speech-to-speech model is offered its tools once, when it +// opens, so it is offered the deferred ones whole instead (offeredDeferred). +func (a *Agent) defers() bool { + return len(a.tools().Deferred) > 0 && !a.native() +} + +// offeredDeferred is what the model is offered for its deferred tools: search_tool, +// load_tool and the tools it has loaded, in the order it loaded them, so a load adds to the +// end of what it is offered and leaves the rest in place. A speech-to-speech model gets them +// all. +func (a *Agent) offeredDeferred() []harness.Tool { + deferred := a.tools().Deferred + if len(deferred) == 0 { + return nil + } + if a.native() { + return deferred + } + offered := []harness.Tool{harness.SearchTool(), harness.LoadTool()} + for _, name := range a.loaded() { + if at := slices.IndexFunc(deferred, func(tool harness.Tool) bool { return tool.Name == name }); at >= 0 { + offered = append(offered, deferred[at]) + } + } + return offered +} + +// runDeferred answers search_tool and load_tool. +func (a *Agent) runDeferred(call llm.ToolCall) (string, error) { + deferred := a.tools().Deferred + if call.Name == harness.SearchToolName { + var arguments struct { + Query string `json:"query"` + } + if err := json.Unmarshal([]byte(call.Arguments), &arguments); err != nil { + return "", fmt.Errorf("agent: could not read the search_tool arguments: %w", err) + } + return harness.SearchDeferred(deferred, arguments.Query), nil + } + result, found := harness.LoadDeferred(deferred, harness.LoadedNames(call.Arguments)) + if err := a.load(found); err != nil { + return "", err + } + return result, nil +} + +// load adds names to the tools the model has loaded and offers them from its next request, +// the next step of the turn that loaded them included: the harness reads its tools on every +// request, so they are set on it directly rather than through SetTools, which waits for the +// turn to finish and so for the very call that asked. +func (a *Agent) load(names []string) error { + a.toolsMu.Lock() + added := false + for _, name := range names { + if !slices.Contains(a.loadedTools, name) { + a.loadedTools = append(a.loadedTools, name) + added = true + } + } + a.toolsMu.Unlock() + if !added { + return nil + } + a.mu.Lock() + current := a.harness + a.mu.Unlock() + if current == nil { + return nil + } + return current.SetTools(a.availableTools()) +} + +// offerLoaded is load for a conversation carried on, which nothing answers a failure to: +// the tools stay as they were, and the model can load them again. +func (a *Agent) offerLoaded(names []string) { + if err := a.load(names); err != nil { + a.logger.Warn("could not offer the tools the conversation had loaded", "tools", names, "error", err) + } +} + +// loaded are the deferred tools the model has loaded, in the order it loaded them. +func (a *Agent) loaded() []string { + a.toolsMu.RLock() + defer a.toolsMu.RUnlock() + return slices.Clone(a.loadedTools) +} + +// loadedIn are the tools the load_tool calls in history asked for, in the order asked, so a +// conversation carried on is offered what it had loaded. A name that is no deferred tool is +// dropped where it is offered (offeredDeferred). +func loadedIn(history []llm.Message) []string { + var names []string + for _, message := range history { + for _, call := range message.ToolCalls { + if call.Name != harness.LoadToolName { + continue + } + for _, name := range harness.LoadedNames(call.Arguments) { + if !slices.Contains(names, name) { + names = append(names, name) + } + } + } + } + return names +} diff --git a/acceleration/internal/agent/deferred_tools_test.go b/acceleration/internal/agent/deferred_tools_test.go new file mode 100644 index 0000000000..2bafb1ecdf --- /dev/null +++ b/acceleration/internal/agent/deferred_tools_test.go @@ -0,0 +1,146 @@ +package agent + +import ( + "strings" + + "github.com/GetStream/Vision-Agents/acceleration/internal/harness" + "github.com/GetStream/Vision-Agents/acceleration/internal/llm" + "github.com/GetStream/Vision-Agents/acceleration/internal/options" +) + +// defersRefunds gives the agent lookup_order to offer and refund_last_order held back. +func (s *AgentSuite) defersRefunds() { + s.ownsTools("refunded") + s.tools.Deferred = []harness.Tool{{ + Name: "refund_last_order", + Description: "Give the customer their money back for the last order.", + Parameters: map[string]any{"type": "object"}, + }} +} + +func offeredNames(tools []harness.Tool) []string { + names := make([]string, len(tools)) + for i, tool := range tools { + names[i] = tool.Name + } + return names +} + +func toolsOf(requested []llm.Tool) []harness.Tool { + tools := make([]harness.Tool, len(requested)) + for i, tool := range requested { + tools[i] = harness.Tool{Name: tool.Name} + } + return tools +} + +func (s *AgentSuite) TestADeferredToolIsNotOfferedButFindingAndLoadingOneIs() { + s.defersRefunds() + s.joinText() + + offered := offeredNames(s.agent.availableTools().Tools) + + s.NotContains(offered, "refund_last_order") + s.Contains(offered, harness.SearchToolName) + s.Contains(offered, harness.LoadToolName) +} + +func (s *AgentSuite) TestNothingDeferredOffersNoSearchTool() { + s.ownsTools("") + s.joinText() + + s.NotContains(offeredNames(s.agent.availableTools().Tools), harness.SearchToolName) +} + +func (s *AgentSuite) TestADeferredToolTheModelHasNotLoadedDoesNotRun() { + s.defersRefunds() + s.joinText() + + s.model.reply = []string{"Refunding."} + s.model.then = []string{"Done."} + s.asksFor("refund_last_order", `{}`) + + _, err := s.agent.RespondTo(s.ctx, "I want my money back", nil, options.LLM{}) + s.Require().NoError(err) + + s.eventually(func() bool { return len(s.model.requests()) >= 1 }, "the model was never asked") + s.never(func() bool { return len(s.runner.asked()) > 0 }, "a tool the model never loaded ran") +} + +func (s *AgentSuite) TestALoadedToolIsOfferedAfterEveryOther() { + s.defersRefunds() + s.joinText() + before := offeredNames(s.agent.availableTools().Tools) + + s.agent.runTool(harness.ToolRequested{TurnID: "turn-1", Call: llm.ToolCall{ + ID: "call-1", Name: harness.LoadToolName, Arguments: `{"names":["refund_last_order"]}`, + }}) + + s.Equal(append(before, "refund_last_order"), offeredNames(s.agent.availableTools().Tools), + "a load moved the tools offered before it, which costs the provider's cache of them") +} + +func (s *AgentSuite) TestALoadedToolRunsWhenTheModelCallsIt() { + s.defersRefunds() + s.joinText() + s.agent.runTool(harness.ToolRequested{TurnID: "turn-1", Call: llm.ToolCall{ + ID: "call-1", Name: harness.LoadToolName, Arguments: `{"names":["refund_last_order"]}`, + }}) + // The load's result is answered by a reply of its own; the call is the one after it. + s.eventually(func() bool { return len(s.model.requests()) == 1 }, "the load was never answered") + s.model.mu.Lock() + s.model.calls = []llm.ToolCall{{ID: "call-2", Name: "refund_last_order", Arguments: `{}`}} + s.model.callsOn = 2 + s.model.then = []string{"Done."} + s.model.mu.Unlock() + + _, err := s.agent.RespondTo(s.ctx, "I want my money back", nil, options.LLM{}) + s.Require().NoError(err) + + s.eventually(func() bool { return len(s.runner.asked()) == 1 }, "the loaded tool never ran") + s.Equal("refund_last_order", s.runner.asked()[0].Name) +} + +func (s *AgentSuite) TestTheModelIsOfferedWhatItLoadedOnTheNextStepOfTheSameTurn() { + s.defersRefunds() + s.joinText() + s.model.reply = []string{"Let me find that."} + s.model.then = []string{"Done."} + s.asksFor(harness.LoadToolName, `{"names":["refund_last_order"]}`) + + _, err := s.agent.RespondTo(s.ctx, "I want my money back", nil, options.LLM{}) + s.Require().NoError(err) + + s.eventually(func() bool { return len(s.model.requests()) >= 2 }, "the model was never asked after it loaded the tool") + first, second := s.model.requests()[0], s.model.requests()[1] + s.NotContains(offeredNames(toolsOf(first.Tools)), "refund_last_order") + s.Equal("refund_last_order", second.Tools[len(second.Tools)-1].Name) +} + +func (s *AgentSuite) TestASearchAnswersWithTheDeferredToolsItMatches() { + s.defersRefunds() + s.joinText() + + s.agent.runTool(harness.ToolRequested{TurnID: "turn-1", Call: llm.ToolCall{ + ID: "call-1", Name: harness.SearchToolName, Arguments: `{"query":"money back"}`, + }}) + + s.Contains(s.historyWith(llm.ToolResult), `"name":"refund_last_order"`) +} + +func (s *AgentSuite) TestAConversationRestoredIsOfferedTheToolsItHadLoaded() { + s.defersRefunds() + s.joinText() + + s.agent.RestoreHistory([]llm.Message{ + {Role: llm.User, Content: "I want my money back"}, + {Role: llm.Assistant, ToolCalls: []llm.ToolCall{{ + ID: "call-1", Name: harness.LoadToolName, Arguments: `{"names":["refund_last_order"]}`, + }}}, + {Role: llm.ToolResult, ToolCallID: "call-1", Content: `{"loaded":[{"name":"refund_last_order"}]}`}, + }) + + offered := offeredNames(s.agent.availableTools().Tools) + s.Equal("refund_last_order", offered[len(offered)-1]) + s.False(strings.Contains(strings.Join(offered, ","), "refund_last_order,refund_last_order")) +} diff --git a/acceleration/internal/agent/events.go b/acceleration/internal/agent/events.go index dd30a3aa46..3a41fffadf 100644 --- a/acceleration/internal/agent/events.go +++ b/acceleration/internal/agent/events.go @@ -432,6 +432,17 @@ type ToolApprovalDecided struct { func (ToolApprovalDecided) isAgentEvent() {} +// ToolAwaitingApproval is a call held until a person answers its question: on a voice call +// the caller's yes, on a connector binding the user's approval on its step. +type ToolAwaitingApproval struct { + ID string + TurnID string + Tool string + Question string +} + +func (ToolAwaitingApproval) isAgentEvent() {} + type ToolRan struct { ID string TurnID string @@ -443,6 +454,9 @@ type ToolRan struct { // Files are what code the tool ran handed back, published where the person sees them. Files []sandbox.Attachment Err error + // Asked says the call was answered with its approval question and not run: it runs, if + // at all, once the caller says yes (ToolApprovalDecided). + Asked bool } func (ToolRan) isAgentEvent() {} diff --git a/acceleration/internal/agent/telephony.go b/acceleration/internal/agent/telephony.go index 2641a7d2c1..e8233069e1 100644 --- a/acceleration/internal/agent/telephony.go +++ b/acceleration/internal/agent/telephony.go @@ -639,7 +639,8 @@ func (a *Agent) executeTool(ctx context.Context, cancel context.CancelFunc, requ delete(a.refusedCalls, requested.Call.ID) a.mu.Unlock() // ran is whether the tool was called: the router refuses some calls itself, before it. - var ran bool + // asked is whether it was answered with its approval question instead. + var ran, asked bool // What the caller said and the tools returned is checked against the whole call, not the // history the model reads: compaction summarises that, and a value it summarised away // was still said or returned. @@ -678,7 +679,7 @@ func (a *Agent) executeTool(ctx context.Context, cancel context.CancelFunc, requ said, err = a.generateImage(ctx, requested.Call) parts = llm.TextParts(said) } else if question := a.approvalQuestion(tool); question != "" { - parts = a.askApproval(requested.Call, question) + parts, asked = a.askApproval(requested.Call, question), true } else { ran = true parts, left, err = a.callTool(ctx, requested.Call) @@ -724,6 +725,7 @@ func (a *Agent) executeTool(ctx context.Context, cancel context.CancelFunc, requ Result: result, Files: files, Err: err, + Asked: asked, }) // A native model is answered on its own session and carries on by itself; there is no // follow-up turn to queue because the model speaks the moment it has the result. @@ -898,6 +900,11 @@ func (a *Agent) callTool(ctx context.Context, call llm.ToolCall) ([]llm.ContentP return llm.TextParts(result), false, err } + if slices.Contains(harness.DeferredToolNames, call.Name) && a.defers() { + result, err := a.runDeferred(call) + return llm.TextParts(result), false, err + } + if a.options.ToolRunner == nil { return nil, false, fmt.Errorf("agent: %s is not a tool this agent can run", call.Name) } @@ -973,6 +980,8 @@ func (a *Agent) availableTools() harness.Tools { if draws { available.Tools = append(available.Tools, generateImageTool()) } + // Last, so a tool the model loads is added after everything it was offered before. + available.Tools = append(available.Tools, a.offeredDeferred()...) return available } diff --git a/acceleration/internal/api/apierror.go b/acceleration/internal/api/apierror.go index 19f94afd7c..9586cc22bd 100644 --- a/acceleration/internal/api/apierror.go +++ b/acceleration/internal/api/apierror.go @@ -5,6 +5,7 @@ import ( "encoding/json" "net/http" "reflect" + "strings" "github.com/danielgtaylor/huma/v2" ) @@ -121,6 +122,7 @@ const ( codeCallUnanswered = "call_unanswered" codeNoOAuthClient = "no_oauth_client" codeInvalidConfig = "invalid_config" + codeToolNotConfigured = "tool_not_configured" codeConcurrencyLimited = "concurrency_limited" codeModelFailed = "model_failed" codePromptInjection = "prompt_injection" @@ -236,7 +238,7 @@ func (*ErrorResponse) TransformSchema(_ huma.Registry, schema *huma.Schema) *hum type ErrorDetail struct { Message string `json:"message" doc:"What went wrong, for a person to read. Its wording may change; branch on code."` Type ErrorType `json:"type"` - Code string `json:"code" doc:"What went wrong, for a program to branch on. Every type has a code of its own name (invalid_request, unauthenticated, forbidden, payment_required, not_found, method_not_allowed, not_acceptable, conflict, gone, payload_too_large, unsupported_media_type, rate_limited, internal_error, unavailable, timeout) that a failure has when nothing names it better. The others are validation_failed, missing_customer, missing_organization, server_side_only, not_configured (this deployment does not offer the feature), modality_not_routed, unsynced_changes (a sync asked to check would write over somebody's edits), name_taken (a 409: another agent config, router config or voice, or another skill of the same agent config, already has the name, so another name will do), channel_connection_taken (a 409: another live agent config binds the channel connection as fixed, and one agent answers a channel connection's messages, so the message names that config), call_unanswered (a 409: the call placed for a phone session ended with nobody answering, and the message names how: busy, no_answer, failed or canceled. Opening the session is refused, or, when the status came while the agent was joining the call, the session is ended at once and kept as ended), no_oauth_client (a 400: a consent found no OAuth client for the connector, and the message names each way the connector lets the app give it one), invalid_config (a 400: an agent config a run names does not start a session), insufficient_credits (a 402: the organization's credit is spent, so work that costs money is refused until credit is added; a model that costs nothing is still served), system_limit_reached (a 402: the organization's spend hard limit for the day or the month is reached, whatever its credit, and the message names the interval, the amount and when it resets), spend_limit_reached (a 402: one of the organization's own spend limits is spent, on the organization, an app, a key, a usage category or a team, and the message says which and when it resets, so retrying before then is refused again), user_limit_reached (a 429: a spend limit on the one end user the request is for is spent, which stops that user and no one else, and Retry-After says when its window ends), spend_check_unavailable (a 503: the router could not share what the organization is spending with the other routers for a while, so work that costs money is refused until it can; a model that costs nothing is still served, and Retry-After says to try again in a few seconds), budget_moved (a 400: a policy sent a budget, which spend limits replaced), limit_above_parent (a 400: a spend limit is above a limit of its parent in the same interval, and the message names the parent's limit), limit_exists (a 409: the subject already has a limit at that level and interval, and the message names its id), no_limit (a 404: the subject has no spend limit to reset or grant past), use_team (a 400: a category starting team: was sent, where the team field names a team), invalid_expiry (a 400: a grant's expires_at is not in the future, or is more than 366 days away), too_many_grants (a 409: the organization has 20 live grants, as many as it may), tier_limit_reached (the organization's tier limits refuse the work, and limit says which: a 403 for a call or a transfer, to a country the tier does not reach, past its minutes or recipients of the day, or past its calls at once; a 400 for an agent config whose sandbox is bigger or more open than the organization's sandbox limits allow. The message names the tier and the limit), account_suspended (a 403: the organization is suspended, and every request is refused, one for a model that costs nothing included, until it is restored; the message says to contact support), concurrency_limited (a 429: the app has as many runs in flight as it may, and Retry-After says when to try again), model_failed (a 503: the model failed or gave no answer), prompt_injection (a 400: the prompt injection screen refused the input of a run or a tool result it read, which retrying an unchanged input does not fix), image_unavailable (a 400: an image a run was handed could not be fetched or is not a JPEG, PNG, WebP or GIF, which retrying the same image does not fix), document_too_large (a 413: a file sent with a response has more pages or text than one may, alone or with the files the conversation already holds, and the message names the limit), document_unreadable (a 400: a file sent with a response could not be read, such as one with a password, one that is damaged or one with no text in it, which retrying the same file does not fix), invalid_usage_categories (a 400: the usage categories the request names, in the token's usage_categories and team claims or in X-Stream-Usage-Categories, are more than 10, one is not 1 to 64 characters of A-Z, a-z, 0-9, _ . : -, more than one names a team, or the team is over 59 characters), memory_unavailable (a 503: the memory store could not delete what was asked, so nothing that waits on it was deleted either, and the same request later can succeed), billing_not_enabled (a 404: this deployment bills nobody, or the caller's app is in no organization it bills, so there is no account to read), provider_takes_no_key (a 400: a key was put for a provider whose work is never routed on a customer's own key here, and the message names the ones that are), campaign_locked (a 409: the campaign's state does not allow the change, such as a launched campaign's sequence, or resuming one that is running), campaign_not_ready (a 400: a launch found the campaign missing what it needs to ring anybody, and the message says what), consent_required (a 400: a launch did not say the people on the campaign agreed to be called), branch_limit_reached (a 409: the conversation has as many edit and regenerate points as Stream Chat keeps for it, 100, so a regenerate or an edit is refused and the conversation is continued in a new chat; asking and reading are unaffected), branch_too_long (a 409: the branch an activation would bring back is more than 100 messages to send again, so it is continued in a new chat), branch_unavailable (a 410: the copies of a hidden branch were deleted, as a memory truncate deletes them, so it can be neither read nor brought back), switch_unfinished (a 409: a regenerate, an edit or an activation of the session failed after Stream Chat may have rewound, and finishing it still fails because Stream Chat does not answer, so the session's questions, regenerates, edits and activations are refused meanwhile; try again: each one tries to finish it first, so it clears once Stream Chat answers), and _not_found for agent_config, call, campaign, channel_account, command, connection, grant, knowledge_document, knowledge_url, limit, provider_key, router_config, scheduled_task, session, simulation, simulation_run, skill and voice. A scheduled task also has invalid_schedule (a schedule that does not read: at and in both or neither, a date, time or duration that does not parse, or until without repeat), schedule_in_past, schedule_too_far (a one-shot, or an in, more than a year ahead), invalid_timezone, timezone_unknown (no zone was given and neither the user nor the config has one), invalid_repeat, interval_too_short (a run moved to less than 15 minutes from now and earlier than the run it replaces) and too_many_tasks (a 409: the user is at their active-task limit). More may be added, so a client should expect one it does not know."` + Code string `json:"code" doc:"What went wrong, for a program to branch on. Every type has a code of its own name (invalid_request, unauthenticated, forbidden, payment_required, not_found, method_not_allowed, not_acceptable, conflict, gone, payload_too_large, unsupported_media_type, rate_limited, internal_error, unavailable, timeout) that a failure has when nothing names it better. The others are validation_failed, missing_customer, missing_organization, server_side_only, not_configured (this deployment does not offer the feature), modality_not_routed, unsynced_changes (a sync asked to check would write over somebody's edits), name_taken (a 409: another agent config, router config or voice, or another skill of the same agent config, already has the name, so another name will do), channel_connection_taken (a 409: another live agent config binds the channel connection as fixed, and one agent answers a channel connection's messages, so the message names that config), call_unanswered (a 409: the call placed for a phone session ended with nobody answering, and the message names how: busy, no_answer, failed or canceled. Opening the session is refused, or, when the status came while the agent was joining the call, the session is ended at once and kept as ended), no_oauth_client (a 400: a consent found no OAuth client for the connector, and the message names each way the connector lets the app give it one), invalid_config (a 400: an agent config a run names does not start a session), tool_not_configured (a 400: a session under an agent config was declared or given a function that none of the config's tool groups names, and the message names it), insufficient_credits (a 402: the organization's credit is spent, so work that costs money is refused until credit is added; a model that costs nothing is still served), system_limit_reached (a 402: the organization's spend hard limit for the day or the month is reached, whatever its credit, and the message names the interval, the amount and when it resets), spend_limit_reached (a 402: one of the organization's own spend limits is spent, on the organization, an app, a key, a usage category or a team, and the message says which and when it resets, so retrying before then is refused again), user_limit_reached (a 429: a spend limit on the one end user the request is for is spent, which stops that user and no one else, and Retry-After says when its window ends), spend_check_unavailable (a 503: the router could not share what the organization is spending with the other routers for a while, so work that costs money is refused until it can; a model that costs nothing is still served, and Retry-After says to try again in a few seconds), budget_moved (a 400: a policy sent a budget, which spend limits replaced), limit_above_parent (a 400: a spend limit is above a limit of its parent in the same interval, and the message names the parent's limit), limit_exists (a 409: the subject already has a limit at that level and interval, and the message names its id), no_limit (a 404: the subject has no spend limit to reset or grant past), use_team (a 400: a category starting team: was sent, where the team field names a team), invalid_expiry (a 400: a grant's expires_at is not in the future, or is more than 366 days away), too_many_grants (a 409: the organization has 20 live grants, as many as it may), tier_limit_reached (the organization's tier limits refuse the work, and limit says which: a 403 for a call or a transfer, to a country the tier does not reach, past its minutes or recipients of the day, or past its calls at once; a 400 for an agent config whose sandbox is bigger or more open than the organization's sandbox limits allow. The message names the tier and the limit), account_suspended (a 403: the organization is suspended, and every request is refused, one for a model that costs nothing included, until it is restored; the message says to contact support), concurrency_limited (a 429: the app has as many runs in flight as it may, and Retry-After says when to try again), model_failed (a 503: the model failed or gave no answer), prompt_injection (a 400: the prompt injection screen refused the input of a run or a tool result it read, which retrying an unchanged input does not fix), image_unavailable (a 400: an image a run was handed could not be fetched or is not a JPEG, PNG, WebP or GIF, which retrying the same image does not fix), document_too_large (a 413: a file sent with a response has more pages or text than one may, alone or with the files the conversation already holds, and the message names the limit), document_unreadable (a 400: a file sent with a response could not be read, such as one with a password, one that is damaged or one with no text in it, which retrying the same file does not fix), invalid_usage_categories (a 400: the usage categories the request names, in the token's usage_categories and team claims or in X-Stream-Usage-Categories, are more than 10, one is not 1 to 64 characters of A-Z, a-z, 0-9, _ . : -, more than one names a team, or the team is over 59 characters), memory_unavailable (a 503: the memory store could not delete what was asked, so nothing that waits on it was deleted either, and the same request later can succeed), billing_not_enabled (a 404: this deployment bills nobody, or the caller's app is in no organization it bills, so there is no account to read), provider_takes_no_key (a 400: a key was put for a provider whose work is never routed on a customer's own key here, and the message names the ones that are), campaign_locked (a 409: the campaign's state does not allow the change, such as a launched campaign's sequence, or resuming one that is running), campaign_not_ready (a 400: a launch found the campaign missing what it needs to ring anybody, and the message says what), consent_required (a 400: a launch did not say the people on the campaign agreed to be called), branch_limit_reached (a 409: the conversation has as many edit and regenerate points as Stream Chat keeps for it, 100, so a regenerate or an edit is refused and the conversation is continued in a new chat; asking and reading are unaffected), branch_too_long (a 409: the branch an activation would bring back is more than 100 messages to send again, so it is continued in a new chat), branch_unavailable (a 410: the copies of a hidden branch were deleted, as a memory truncate deletes them, so it can be neither read nor brought back), switch_unfinished (a 409: a regenerate, an edit or an activation of the session failed after Stream Chat may have rewound, and finishing it still fails because Stream Chat does not answer, so the session's questions, regenerates, edits and activations are refused meanwhile; try again: each one tries to finish it first, so it clears once Stream Chat answers), and _not_found for agent_config, call, campaign, channel_account, command, connection, grant, knowledge_document, knowledge_url, limit, provider_key, router_config, scheduled_task, session, simulation, simulation_run, skill and voice. A scheduled task also has invalid_schedule (a schedule that does not read: at and in both or neither, a date, time or duration that does not parse, or until without repeat), schedule_in_past, schedule_too_far (a one-shot, or an in, more than a year ahead), invalid_timezone, timezone_unknown (no zone was given and neither the user nor the config has one), invalid_repeat, interval_too_short (a run moved to less than 15 minutes from now and earlier than the run it replaces) and too_many_tasks (a 409: the user is at their active-task limit). More may be added, so a client should expect one it does not know."` DocURL string `json:"doc_url" format:"uri" doc:"Where the code is explained."` Limit string `json:"limit,omitempty" doc:"Which limit refused the work, present with tier_limit_reached only: country, per_day, recipients_per_day or concurrent for a call or a transfer, sandbox_network, sandbox_cpu, sandbox_memory_gb or sandbox_disk_gb for an agent config's sandbox. More may be added."` } @@ -300,6 +302,13 @@ func notConfigured(message string) APIError { return APIError{Type: ErrorTypeInvalidRequest, Code: codeNotConfigured, Message: message} } +// toolNotConfigured is a function a session under an agent config was asked to offer that +// none of the config's tool groups names. +func toolNotConfigured(err error) APIError { + return APIError{Type: ErrorTypeInvalidRequest, Code: codeToolNotConfigured, + Message: strings.TrimPrefix(err.Error(), "session: ") + ": name it in a tool group under tools in agent.yaml or the agent config"} +} + // invalidTimezone is a zone a request named that nobody knows. func invalidTimezone(message string) APIError { return APIError{Type: ErrorTypeInvalidRequest, Code: codeInvalidTimezone, Message: message} diff --git a/acceleration/internal/api/config_patch.go b/acceleration/internal/api/config_patch.go index ca42066365..f663db6934 100644 --- a/acceleration/internal/api/config_patch.go +++ b/acceleration/internal/api/config_patch.go @@ -46,7 +46,7 @@ type AgentConfigPatch struct { Harness *Harness `json:"harness,omitempty"` Dispatch *AgentDispatch `json:"dispatch,omitempty"` EpisodeCards *EpisodeCardsMode `json:"episode_cards,omitempty"` - Tools *AgentTools `json:"tools,omitempty"` + Tools *[]ToolGroup `json:"tools,omitempty" maxItems:"64" doc:"Replaces the tool groups whole; an empty list removes them all."` Scheduling *AgentScheduling `json:"scheduling,omitempty"` Calls *AgentCalls `json:"calls,omitempty"` ResumeHistory *ResumeHistory `json:"resume_history,omitempty"` @@ -233,8 +233,9 @@ func (s *Server) patchAgentConfig(ctx context.Context, request *patchAgentConfig if patch.EpisodeCards != nil { setEpisodeCards(&config, *patch.EpisodeCards) } - config.ProgressiveTools = progressiveOf(config.ProgressiveTools, patch.Tools) - config.ToolOutputRetention = outputRetentionOf(config.ToolOutputRetention, patch.Tools) + if patch.Tools != nil { + config.ToolGroups = toolGroupsOf(*patch.Tools) + } config.Scheduling = schedulingOr(config.Scheduling, patch.Scheduling) config.Calls = callsOr(config.Calls, patch.Calls) config.ResumeHistory = resumeHistoryOr(config.ResumeHistory, patch.ResumeHistory) @@ -251,7 +252,7 @@ func (s *Server) patchAgentConfig(ctx context.Context, request *patchAgentConfig if message, ok := callsComplaint(config.Calls); !ok { return nil, invalidRequest(message) } - if message, ok := outputRetentionComplaint(config); !ok { + if message, ok := toolsComplaint(config); !ok { return nil, invalidRequest(message) } config.Tags = override(config.Tags, patch.Tags) diff --git a/acceleration/internal/api/configs.go b/acceleration/internal/api/configs.go index 45fc4f2787..5003395eab 100644 --- a/acceleration/internal/api/configs.go +++ b/acceleration/internal/api/configs.go @@ -170,7 +170,7 @@ func (s *Server) createAgentConfig(ctx context.Context, request *createAgentConf if message, ok := teamTagComplaint(config.Tags, nil); !ok { return nil, invalidRequest(message) } - if message, ok := outputRetentionComplaint(config); !ok { + if message, ok := toolsComplaint(config); !ok { return nil, invalidRequest(message) } if message, ok := s.channelsComplaint(ctx, config); !ok { @@ -306,8 +306,11 @@ func (s *Server) updateAgentConfig(ctx context.Context, request *updateAgentConf } // Only a sync writes them, so a save from the dashboard keeps them. config.LeftOutConnectors = existing.LeftOutConnectors - config.ProgressiveTools = progressiveOf(existing.ProgressiveTools, request.Body.Tools) - config.ToolOutputRetention = outputRetentionOf(existing.ToolOutputRetention, request.Body.Tools) + // Kept for the same reason: a client that does not know the groups must not take away + // every function the agent offers by saving. + if request.Body.Tools == nil { + config.ToolGroups = existing.ToolGroups + } config.ReviewInstructions = reviewOf(existing.ReviewInstructions, request.Body.Review) config.UIInstructions = override(existing.UIInstructions, request.Body.UiInstructions) // Field by field, as the doc says of an update: a block naming only the limit keeps @@ -331,7 +334,7 @@ func (s *Server) updateAgentConfig(ctx context.Context, request *updateAgentConf if message, ok := teamTagComplaint(config.Tags, existing.Tags); !ok { return nil, invalidRequest(message) } - if message, ok := outputRetentionComplaint(config); !ok { + if message, ok := toolsComplaint(config); !ok { return nil, invalidRequest(message) } if message, ok := s.channelsComplaint(ctx, config); !ok { @@ -1140,6 +1143,8 @@ func storedBindings(bindings []AgentConnectorBinding) []store.ConnectorBinding { Policy: bindingPolicyOf(binding.Policy), Sandbox: binding.Sandbox, RequireApproval: binding.RequireApproval, + Loading: string(value(binding.Loading)), + OutputRetention: options.Retention(value(binding.OutputRetention)), }) } return stored @@ -1203,6 +1208,11 @@ func bindingsOf(bindings []store.ConnectorBinding) []AgentConnectorBinding { Required: &required, Sandbox: binding.Sandbox, RequireApproval: binding.RequireApproval, + OutputRetention: optional(string(binding.OutputRetention)), + } + if binding.Loading != "" { + loading := ToolLoading(binding.Loading) + one.Loading = &loading } if binding.TimeoutMs > 0 { timeout := binding.TimeoutMs @@ -1621,8 +1631,7 @@ func storedConfig(request AgentConfigRequest, customerID string) store.AgentConf setEpisodeCards(&config, *request.EpisodeCards) } if request.Tools != nil { - config.ProgressiveTools = value(request.Tools.Progressive) - config.ToolOutputRetention = options.Retention(value(request.Tools.OutputRetention)) + config.ToolGroups = toolGroupsOf(*request.Tools) } config.Scheduling = schedulingOf(request.Scheduling) config.Calls = callsOf(request.Calls) @@ -1720,8 +1729,7 @@ func agentConfigOf(config store.AgentConfig) AgentConfig { rendered.Dispatch = dispatchOf(config) episodeCards := episodeCardsOf(config) rendered.EpisodeCards = &episodeCards - progressive := config.ProgressiveTools - rendered.Tools = &AgentTools{Progressive: &progressive, OutputRetention: optional(string(config.ToolOutputRetention))} + rendered.Tools = renderedToolGroups(config.ToolGroups) rendered.Scheduling = renderedScheduling(config.Scheduling) rendered.Calls = renderedCalls(config.Calls) rendered.ResumeHistory = renderedResumeHistory(config.ResumeHistory) @@ -2005,7 +2013,7 @@ type AgentConfigRequest struct { Region *string `json:"region,omitempty" pattern:"^[a-z][a-z0-9-]{0,31}$" patternDescription:"a letter and up to 31 of a-z, 0-9 and hyphens, such as us or eu" example:"eu" doc:"Where the agent's work is done: us or eu for now. Every model its sessions run, its knowledge base and its sandbox are in it, and none of its work is served in another region. A config naming a model with no instance there, a knowledge_namespace kept in another region, or a sandbox where none runs there is refused (400), naming each: a knowledge base is never moved. Left out, a new agent is in us and an update keeps the stored region."` Role *AgentRole `json:"role,omitempty"` Review *AgentReview `json:"review,omitempty" doc:"How a finished call under this agent is reviewed. Left out on an update, the stored instructions stay."` - Tools *AgentTools `json:"tools,omitempty" doc:"How the agent is offered its connector tools. Left out on an update, the stored settings stay."` + Tools *[]ToolGroup `json:"tools,omitempty" maxItems:"64" doc:"The functions an SDK provides that the agent may offer, in named groups, each with how it is loaded. A function an SDK registers, declares on a session or hosts in a worker is offered only when a group names it: a session declaring one that none names is refused (400 tool_not_configured), and a worker's is left out. The router's own tools, connector tools and other agents are not named here. Omitted or null on an update, the groups stored stay; an empty list removes them all. The mapping this field once was, with progressive and output_retention, is refused: loading summary and output_retention on a group or a connector binding replaced them."` Scheduling *AgentScheduling `json:"scheduling,omitempty" doc:"Whether the agent may schedule tasks for its users, and the defaults those tasks take. Left out on an update, the stored settings stay."` Calls *AgentCalls `json:"calls,omitempty" doc:"The tools an agent on a phone call is offered: end the call, press keys, transfer. Left out, transfer and press are on and end_call is off. On an update, a tool left out keeps its stored settings and a tool sent replaces its settings whole."` ResumeHistory *ResumeHistory `json:"resume_history,omitempty" doc:"How a conversation is handed back to the model when a session under this agent carries it on: reopened, resumed by id, joined by voice or forked. Each field set here wins over the app's and the organization's policy, and a session request's wins over this; a field set nowhere is the router's default: replies as text, the newest 100 turns within 60000 characters. Left out on an update, the stored settings stay."` @@ -2053,7 +2061,7 @@ type AgentConfig struct { Region string `json:"region" example:"us" doc:"Where the agent's work is done: every model its sessions run, its knowledge base and its sandbox are in it. us unless the agent's agent.yaml or config names another."` Role *AgentRole `json:"role,omitempty"` Review *AgentReview `json:"review,omitempty" doc:"Absent when calls are reviewed by Stream's default guide."` - Tools *AgentTools `json:"tools,omitempty"` + Tools *[]ToolGroup `json:"tools,omitempty" doc:"The functions an SDK provides that the agent may offer, in named groups. Empty when there are none, and then the agent offers no function."` Scheduling *AgentScheduling `json:"scheduling,omitempty"` Calls *AgentCalls `json:"calls,omitempty"` ResumeHistory *ResumeHistory `json:"resume_history,omitempty"` @@ -2079,17 +2087,109 @@ type AgentConfig struct { Voice *string `json:"voice,omitempty" doc:"The voice the agent speaks in, as TtsOptions.voice takes it: a provider's own voice id, or one of your voices, with the custom: prefix to mean only yours. A native agent (sts) takes the vendor's own voice name instead."` } -// AgentTools is how an agent is offered its plugin, MCP server and connector tools. -type AgentTools struct { - Progressive *bool `json:"progressive,omitempty" doc:"Offer each tool by the first line of its description, with its arguments' descriptions left out, and have the first call to a tool return its full description and input schema instead of running it. It saves context on an agent with many tools, at the cost of one more model turn for each tool a conversation uses. Off by default. Left out on an update, the stored setting stays."` - OutputRetention *string `json:"output_retention,omitempty" pattern:"^(|none|[1-9][0-9]*[hd])$" patternDescription:"none, a number of hours such as 12h, or a number of days such as 30d" example:"30d" doc:"How long this agent's sessions keep what any of its tools were given and gave back on the turns they record: a tool's arguments, its result, a search's query and the wording of a tool's error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed. Left out on a create, or empty, it is kept as long as the session; left out on an update, the stored setting stays, and an empty string removes it. The app's and the organization's policies (tool_output_retention) can shorten it and never lengthen it, and the shortest applies. It applies to tool calls recorded after it is set."` -} +// ToolLoading is how a tool is offered to the model. +type ToolLoading string -func (*AgentTools) TransformSchema(_ huma.Registry, schema *huma.Schema) *huma.Schema { - schema.Description = "How an agent is offered its tools, and how long what they were given and gave back is kept." +const ( + ToolLoadingFull ToolLoading = store.ToolLoadingFull + ToolLoadingSummary ToolLoading = store.ToolLoadingSummary + ToolLoadingDeferred ToolLoading = store.ToolLoadingDeferred +) + +func (ToolLoading) Schema(registry huma.Registry) *huma.Schema { + ref := namedEnum(registry, "ToolLoading", "How a tool is offered to the model. full sends its whole "+ + "description and input schema from the start. summary sends the first line of its description and "+ + "its arguments without their descriptions, and the first call returns the full description and "+ + "schema instead of running it: one more model turn for each tool a conversation uses, for less "+ + "context on every turn. deferred offers nothing until the model needs it: an agent with a deferred "+ + "tool is offered search_tool, which finds deferred tools by what they do, and load_tool, which "+ + "returns their full descriptions and offers them from the model's next step on.", + string(ToolLoadingFull), string(ToolLoadingSummary), string(ToolLoadingDeferred)) + registry.Map()["ToolLoading"].Default = string(ToolLoadingFull) + return ref +} + +// ToolGroup is a named set of functions an SDK provides that an agent may offer. +type ToolGroup struct { + Name string `json:"name" pattern:"^[a-z][a-z0-9_-]{0,62}$" patternDescription:"a lowercase letter, then up to 62 lowercase letters, digits, - or _" doc:"What the group is called, unique within the config."` + Loading *ToolLoading `json:"loading,omitempty"` + Tools []string `json:"tools" minItems:"1" maxItems:"128" nullable:"false" doc:"The functions' names, as the SDK registers them. A name sits in one group at most."` + OutputRetention *string `json:"output_retention,omitempty" pattern:"^(|none|[1-9][0-9]*[hd])$" patternDescription:"none, a number of hours such as 12h, or a number of days such as 30d" example:"30d" doc:"How long the agent's sessions keep what the group's tools were given and gave back on the turns they record: a tool's arguments, its result and the wording of its error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed. Left out or empty, it is kept as long as the session. The app's and the organization's policies (tool_output_retention) can shorten it and never lengthen it. It applies to tool calls recorded after it is set."` +} + +func (*ToolGroup) TransformSchema(_ huma.Registry, schema *huma.Schema) *huma.Schema { + schema.Description = "A named set of functions an SDK provides that the agent may offer, how they are " + + "loaded and how long what they were given and gave back is kept. A function an SDK registers, " + + "declares on a session or hosts in a worker is offered only when a group names it." return schema } +// toolGroupsOf turns the groups a caller sent into what a config stores. +func toolGroupsOf(groups []ToolGroup) []store.ToolGroup { + stored := make([]store.ToolGroup, 0, len(groups)) + for _, group := range groups { + stored = append(stored, store.ToolGroup{ + Name: group.Name, + Loading: string(value(group.Loading)), + Tools: slices.Clone(group.Tools), + OutputRetention: options.Retention(value(group.OutputRetention)), + }) + } + return stored +} + +// renderedToolGroups renders a config's groups for the wire, nil when there are none. +func renderedToolGroups(groups []store.ToolGroup) *[]ToolGroup { + rendered := make([]ToolGroup, 0, len(groups)) + for _, group := range groups { + loading := ToolLoading(cmp.Or(group.Loading, store.ToolLoadingFull)) + rendered = append(rendered, ToolGroup{ + Name: group.Name, + Loading: &loading, + Tools: slices.Clone(group.Tools), + OutputRetention: optional(string(group.OutputRetention)), + }) + } + return &rendered +} + +// toolsComplaint refuses what the schema of a config's tool groups and connector bindings +// cannot say: two groups by one name, a function named in two groups or by a name the router +// keeps for itself, and a retention out of bounds. It reads the config as it will be stored, +// so a create, an update, a patch and a sync are held to the same rules. +func toolsComplaint(config store.AgentConfig) (string, bool) { + names := map[string]bool{} + grouped := map[string]string{} + for i, group := range config.ToolGroups { + at := fmt.Sprintf("tools[%d]", i) + if names[group.Name] { + return fmt.Sprintf("%s: two tool groups are called %q", at, group.Name), false + } + names[group.Name] = true + for _, tool := range group.Tools { + name := strings.TrimSpace(tool) + switch { + case name == "": + return at + ".tools: a tool has no name", false + case slices.Contains(harness.DeferredToolNames, name): + return fmt.Sprintf("%s.tools: %s is the router's own tool for loading deferred ones, so no function may take the name", at, name), false + case grouped[name] != "": + return fmt.Sprintf("%s.tools: %s is already in tool group %q, and a tool sits in one group", at, name, grouped[name]), false + } + grouped[name] = group.Name + } + if message, ok := toolOutputRetentionComplaint(at+".output_retention", group.OutputRetention); !ok { + return message, false + } + } + for i, binding := range config.Connectors { + if message, ok := toolOutputRetentionComplaint(fmt.Sprintf("connectors[%d].output_retention", i), binding.OutputRetention); !ok { + return message, false + } + } + return "", true +} + // AgentReview is how a finished call under an agent is reviewed. type AgentReview struct { Instructions *string `json:"instructions,omitempty" doc:"What the model reviewing a finished call judges it by, in place of Stream's default guide: what to summarise and how to score the call from 1 to 5. A score outside 1 to 5 is not kept. The reviewer is always told to answer with JSON holding a summary, a score and notes, so leave the answer's format out. Empty is the default guide. On an update or a patch, left out keeps the stored instructions and an empty string removes them."` @@ -2357,20 +2457,6 @@ func callsComplaint(calls store.AgentCalls) (string, bool) { return "", true } -// outputRetentionOf is the tool output retention once tools is written over stored: what -// it says, or stored when it says nothing. -func outputRetentionOf(stored options.Retention, tools *AgentTools) options.Retention { - if tools == nil || tools.OutputRetention == nil { - return stored - } - return options.Retention(*tools.OutputRetention) -} - -// outputRetentionComplaint refuses a tool output retention out of bounds. -func outputRetentionComplaint(config store.AgentConfig) (string, bool) { - return toolOutputRetentionComplaint("tools.output_retention", config.ToolOutputRetention) -} - // teamTagComplaint refuses a team tag that cannot be a team's name, since the tag is the team // the agent's sessions are counted under (session.TeamTag). One the config already held is let // through, so a config saved before the tag was a team can still be changed in other ways. @@ -2383,15 +2469,6 @@ func teamTagComplaint(tags, prior map[string]string) (string, bool) { session.TeamTag, team, usage.MaxTeamLength), false } -// progressiveOf is whether tools are offered progressively once tools is written over -// stored: what it says, or stored when it says nothing. -func progressiveOf(stored bool, tools *AgentTools) bool { - if tools == nil { - return stored - } - return override(stored, tools.Progressive) -} - // AgentConnectorBinding is a connector whose tools an agent config may call, under an alias. // // The alias pattern is the prototype's connectorAliasPattern (internal/api/connectors.go:54 on @@ -2415,6 +2492,8 @@ type AgentConnectorBinding struct { Events *[]ConnectorBindingEvent `json:"events,omitempty" maxItems:"32" doc:"MCP events the binding's fixed connection is subscribed to, each opening a text conversation from the config when it arrives. Subscribed when the connection is next validated. Only a fixed binding may declare events: a session binding's connection is picked when a session opens, and an event arrives with no session open."` Policy *ConnectorBindingPolicy `json:"policy,omitempty" doc:"How the binding's tool calls behave around speech and interruptions. Omitted, a call is cancelled at the provider when the turn is interrupted, and the agent picks its own words while it runs."` Sandbox bool `json:"sandbox,omitempty" doc:"Whether code in the session's sandbox may call the connector's API with the binding's connection. The sandbox gets the env var the connector's sandbox block names, holding a placeholder that the sandbox's egress swaps for the credential only in a request to one of its hosts. Refused for a connector without a sandbox block, for a config that sets no sandbox, and for two sandbox bindings whose connectors name the same env var. Omitted is false."` + Loading *ToolLoading `json:"loading,omitempty" doc:"How the binding's tools are offered to the model. Omitted is full."` + OutputRetention *string `json:"output_retention,omitempty" pattern:"^(|none|[1-9][0-9]*[hd])$" patternDescription:"none, a number of hours such as 12h, or a number of days such as 30d" example:"30d" doc:"How long the agent's sessions keep what the binding's tools were given and gave back on the turns they record, as a tool group's output_retention. Omitted or empty, it is kept as long as the session."` RequireApproval []string `json:"require_approval,omitempty" maxItems:"128" nullable:"false" doc:"Names or path.Match patterns of the binding's tools whose every call waits for a person to allow it before the router sends it, such as create_issue or delete_*. The step of the call asks the session's user in its conversation, who answers with the session socket's tool_approval. Declined, or not answered in 10 minutes, the call is not sent and the model is told so. A session with nobody to ask, such as an incognito one or one opened by an MCP event, refuses the call at once. Each entry must match a tool the binding grants, or one of its tool_patterns. Refused with sandbox, whose code calls the API with nobody asked. At most 128, the cap tools has."` } diff --git a/acceleration/internal/api/configs_test.go b/acceleration/internal/api/configs_test.go index e3527ad60d..775ffe8104 100644 --- a/acceleration/internal/api/configs_test.go +++ b/acceleration/internal/api/configs_test.go @@ -334,21 +334,67 @@ func (s *ConfigsSuite) storedCards(id string, on, record bool) { s.Equal(record, stored.EpisodeCardsRecord, "episode_cards_record") } -// Tools are offered whole unless a config asks for them progressively, and an update that -// leaves the setting out keeps it. -func (s *ConfigsSuite) TestAConfigOffersToolsWholeUnlessItSaysProgressively() { +// A config names its functions in tool groups, each offered whole unless it says otherwise, +// and an update that leaves the groups out keeps them. +func (s *ConfigsSuite) TestAConfigNamesItsToolsInGroups() { unnamed := s.createConfig(map[string]any{"name": "support"}) - s.False(value(unnamed.Tools.Progressive)) + s.Require().NotNil(unnamed.Tools) + s.Empty(*unnamed.Tools, "a config names no function until it says so") var patched AgentConfig - s.Require().Equal(http.StatusOK, s.serverClient.do(http.MethodPatch, - "/v1/agents/configs/"+unnamed.Id, map[string]any{"tools": map[string]any{"progressive": true}}, &patched)) - s.True(value(patched.Tools.Progressive)) + s.Require().Equal(http.StatusOK, s.serverClient.do(http.MethodPatch, "/v1/agents/configs/"+unnamed.Id, + map[string]any{"tools": []any{ + map[string]any{"name": "booking", "tools": []any{"lookup_booking"}}, + map[string]any{"name": "refunds", "loading": "deferred", "tools": []any{"refund_booking", "void_ticket"}}, + }}, &patched)) + s.Require().NotNil(patched.Tools) + s.Require().Len(*patched.Tools, 2) + s.Equal(ToolLoadingFull, value((*patched.Tools)[0].Loading), "a group that says nothing is offered whole") + s.Equal(ToolLoadingDeferred, value((*patched.Tools)[1].Loading)) + s.Equal([]string{"refund_booking", "void_ticket"}, (*patched.Tools)[1].Tools) var saved AgentConfig s.Require().Equal(http.StatusOK, s.serverClient.do(http.MethodPut, - "/v1/agents/configs/"+unnamed.Id, map[string]any{"name": "support", "tools": map[string]any{}}, &saved)) - s.True(value(saved.Tools.Progressive), "a setting left out on an update stays") + "/v1/agents/configs/"+unnamed.Id, map[string]any{"name": "support"}, &saved)) + s.Require().NotNil(saved.Tools, "groups left out on an update stay") + s.Len(*saved.Tools, 2) +} + +func (s *ConfigsSuite) TestAToolInTwoGroupsIsRefused() { + status, failure := s.serverClient.failure(http.MethodPost, "/v1/agents/configs", map[string]any{ + "name": "support", "tools": []any{ + map[string]any{"name": "booking", "tools": []any{"lookup_booking"}}, + map[string]any{"name": "lookups", "loading": "summary", "tools": []any{"lookup_booking"}}, + }, + }) + + s.Equal(http.StatusBadRequest, status) + s.Contains(failure, "lookup_booking is already in tool group") +} + +func (s *ConfigsSuite) TestAToolGroupWithALoadingThereIsNoneOfIsRefused() { + status, failure := s.serverClient.failure(http.MethodPost, "/v1/agents/configs", map[string]any{ + "name": "support", "tools": []any{map[string]any{"name": "booking", "loading": "lazy", "tools": []any{"lookup_booking"}}}, + }) + + s.Equal(http.StatusBadRequest, status) + s.Contains(failure, "loading") +} + +func (s *ConfigsSuite) TestAFunctionTakingTheNameOfTheRoutersLoaderIsRefused() { + status, failure := s.serverClient.failure(http.MethodPost, "/v1/agents/configs", map[string]any{ + "name": "support", "tools": []any{map[string]any{"name": "mine", "tools": []any{"load_tool"}}}, + }) + + s.Equal(http.StatusBadRequest, status) + s.Contains(failure, "load_tool") +} + +func (s *ConfigsSuite) TestTheToolSettingsOfBeforeGroupsAreRefused() { + status, _ := s.serverClient.failure(http.MethodPost, "/v1/agents/configs", + map[string]any{"name": "support", "tools": map[string]any{"progressive": true}}) + + s.Equal(http.StatusBadRequest, status) } func (s *ConfigsSuite) TestAConfigRemembersItsScheduling() { @@ -2640,47 +2686,35 @@ func (s *ConfigsSuite) TestAnUpdateKeepingABindingToAForceDeletedConnectorIsAcce s.Equal(http.StatusOK, status) } -// Tool output is kept as long as the session unless a config says otherwise, an update -// that leaves the setting out keeps it, and an empty one removes it. -func (s *ConfigsSuite) TestAConfigKeepsToolOutputForAsLongAsItSays() { - unnamed := s.createConfig(map[string]any{"name": "support"}) - s.Nil(unnamed.Tools.OutputRetention) - - created := s.createConfig(map[string]any{"name": "billing", "tools": map[string]any{"output_retention": "7d"}}) - s.Equal("7d", value(created.Tools.OutputRetention)) - - var saved AgentConfig - s.Require().Equal(http.StatusOK, s.serverClient.do(http.MethodPut, - "/v1/agents/configs/"+created.Id, map[string]any{"name": "billing", "tools": map[string]any{"progressive": true}}, &saved)) - s.Equal("7d", value(saved.Tools.OutputRetention), "a setting left out on an update stays") - - var patched AgentConfig - s.Require().Equal(http.StatusOK, s.serverClient.do(http.MethodPatch, - "/v1/agents/configs/"+created.Id, map[string]any{"tools": map[string]any{"output_retention": "none"}}, &patched)) - s.Equal("none", value(patched.Tools.OutputRetention)) +// A group keeps its tools' output as long as it says, and a group that says nothing as long +// as the session. +func (s *ConfigsSuite) TestAToolGroupKeepsItsOutputForAsLongAsItSays() { + created := s.createConfig(map[string]any{"name": "billing", "tools": []any{ + map[string]any{"name": "cards", "output_retention": "7d", "tools": []any{"charge_card"}}, + map[string]any{"name": "lookups", "tools": []any{"lookup_invoice"}}, + }}) - var cleared AgentConfig - s.Require().Equal(http.StatusOK, s.serverClient.do(http.MethodPatch, - "/v1/agents/configs/"+created.Id, map[string]any{"tools": map[string]any{"output_retention": ""}}, &cleared)) - s.Nil(cleared.Tools.OutputRetention) + s.Require().NotNil(created.Tools) + s.Equal("7d", value((*created.Tools)[0].OutputRetention)) + s.Nil((*created.Tools)[1].OutputRetention) } func (s *ConfigsSuite) TestAToolOutputRetentionOutsideItsBoundsIsRefusedOnEveryWrite() { created := s.createConfig(map[string]any{"name": "support"}) - tooLong := map[string]any{"output_retention": "3651d"} + tooLong := []any{map[string]any{"name": "cards", "output_retention": "3651d", "tools": []any{"charge_card"}}} status, failure := s.serverClient.failure(http.MethodPost, "/v1/agents/configs", map[string]any{"name": "billing", "tools": tooLong}) s.Equal(http.StatusBadRequest, status) - s.Contains(failure, "tools.output_retention") + s.Contains(failure, "tools[0].output_retention") status, failure = s.serverClient.failure(http.MethodPut, "/v1/agents/configs/"+created.Id, map[string]any{"name": "support", "tools": tooLong}) s.Equal(http.StatusBadRequest, status) - s.Contains(failure, "tools.output_retention") + s.Contains(failure, "tools[0].output_retention") status, failure = s.serverClient.failure(http.MethodPatch, "/v1/agents/configs/"+created.Id, map[string]any{"tools": tooLong}) s.Equal(http.StatusBadRequest, status) - s.Contains(failure, "tools.output_retention") + s.Contains(failure, "tools[0].output_retention") } func callsExample() map[string]any { diff --git a/acceleration/internal/api/dispatch_test.go b/acceleration/internal/api/dispatch_test.go index 0daac48b8c..54c3d0b9c8 100644 --- a/acceleration/internal/api/dispatch_test.go +++ b/acceleration/internal/api/dispatch_test.go @@ -383,8 +383,31 @@ func (s *DispatchSuite) TestHostedToolsNeedAnAgentToBeFor() { })) s.Equal("hosting_refused", s.next(worker)["type"]) - tools, _ := s.dispatch.HostedTools(s.customerID(), "") - s.Empty(tools) + tools, _ := s.dispatch.HostedTools(s.customerID(), dispatch.EveryAgent) + s.Empty(tools, "an agent id left out offered the tools to every agent") + + s.Require().NoError(worker.WriteJSON(frame{ + "type": "host_tools", "agent_id": "stream-support", "every_agent": true, + "tools": []frame{{"name": "investigate_sdk", "description": "Read SDK source"}}, + })) + s.Equal("hosting_refused", s.next(worker)["type"]) +} + +func (s *DispatchSuite) TestToolsHostedForEveryAgentAreTheApps() { + worker := s.worker() + s.Require().NoError(worker.WriteJSON(frame{ + "type": "host_tools", "every_agent": true, + "tools": []frame{{"name": "weather_lookup", "description": "Look up the weather"}}, + })) + reply := s.next(worker) + s.Equal("hosting", reply["type"], reply) + s.Equal(true, reply["every_agent"]) + + tools, _ := s.dispatch.HostedTools(s.customerID(), dispatch.EveryAgent) + s.Require().Len(tools, 1) + s.Equal("weather_lookup", tools[0].Name) + tools, _ = s.dispatch.HostedTools(s.customerID(), "stream-support") + s.Empty(tools, "tools hosted for every agent are not one agent's own") } func (s *DispatchSuite) TestAHostedToolCallIsAnsweredOverTheSocket() { diff --git a/acceleration/internal/api/dispatchws.go b/acceleration/internal/api/dispatchws.go index 1aff0326a8..582d3274b0 100644 --- a/acceleration/internal/api/dispatchws.go +++ b/acceleration/internal/api/dispatchws.go @@ -259,10 +259,12 @@ func (s *Server) readWorker(connection *websocket.Conn, worker *dispatch.Worker, // At echoes back a ping, so the worker can measure the round trip itself. At float64 `json:"at"` // AgentID, Tools and TimeoutMs are what a worker offers to run for every - // session opened under one agent id. - AgentID string `json:"agent_id"` - Tools []hostedTool `json:"tools"` - TimeoutMs int `json:"timeout_ms"` + // session opened under one agent id, or with EveryAgent instead of an agent + // id, for every agent of the app whose tool groups name them. + AgentID string `json:"agent_id"` + EveryAgent bool `json:"every_agent"` + Tools []hostedTool `json:"tools"` + TimeoutMs int `json:"timeout_ms"` // ToolCallID, Output and Error answer one hosted tool call, as a session's // tool_result does: Output is a string or a parts array. ID is ToolCallID under // the name workers sent it by before. @@ -307,7 +309,7 @@ func (s *Server) readWorker(connection *websocket.Conn, worker *dispatch.Worker, } case "host_tools": - reply := s.hostTools(worker, report.AgentID, report.Tools, report.TimeoutMs) + reply := s.hostTools(worker, report.AgentID, report.EveryAgent, report.Tools, report.TimeoutMs) select { case replies <- reply: case <-writerDone: @@ -398,10 +400,17 @@ type hostedApproval struct { // is trusted with its customer's conversations already, and naming an agent is not a way // into anybody else's. The agent is named as its config is, since that is the name sessions // are opened by, and a name no config has is refused. -func (s *Server) hostTools(worker *dispatch.Worker, agentID string, declared []hostedTool, timeoutMs int) frame { +func (s *Server) hostTools(worker *dispatch.Worker, agentID string, everyAgent bool, declared []hostedTool, timeoutMs int) frame { refuse := func(reason string) frame { s.logger.Warn("refused a worker's hosted tools", "worker", worker.ID, "agent", agentID, "reason", reason) - return frame{"type": "hosting_refused", "agent_id": agentID, "reason": reason} + return frame{"type": "hosting_refused", "agent_id": agentID, "every_agent": everyAgent, "reason": reason} + } + // An agent id left out by mistake would otherwise offer the tools to every agent. + switch { + case everyAgent && agentID != "": + return refuse("host tools for one agent_id or for every_agent, not both") + case !everyAgent && agentID == "": + return refuse("hosted tools need the agent_id they are for, or every_agent") } tools := make([]dispatch.Tool, 0, len(declared)) names := make([]string, 0, len(declared)) @@ -433,8 +442,8 @@ func (s *Server) hostTools(worker *dispatch.Worker, agentID string, declared []h if s.workers != nil { s.workers.Offer(context.Background(), worker.CustomerID, agentID) } - s.logger.Info("a worker hosts tools", "worker", worker.ID, "agent", agentID, "tools", names) - return frame{"type": "hosting", "agent_id": agentID, "tools": names} + s.logger.Info("a worker hosts tools", "worker", worker.ID, "agent", agentID, "every_agent", everyAgent, "tools", names) + return frame{"type": "hosting", "agent_id": agentID, "every_agent": everyAgent, "tools": names} } // unknownHostedAgent says why tools offered under a name the customer has no agent config diff --git a/acceleration/internal/api/handwritten.go b/acceleration/internal/api/handwritten.go index ca56f50318..81e5a1e6e2 100644 --- a/acceleration/internal/api/handwritten.go +++ b/acceleration/internal/api/handwritten.go @@ -30,6 +30,10 @@ func documentHandWritten(api huma.API) { registry.Schema(reflect.TypeFor[ToolResultCommand](), true, ""), registry.Schema(reflect.TypeFor[ToolApprovalCommand](), true, ""), }, + // The frames the server sends that a client keeps state from. + "x-server-frames": []*huma.Schema{ + registry.Schema(reflect.TypeFor[ToolCallState](), true, ""), + }, }, Method: http.MethodGet, Path: "/v1/agents/sessions/{id}/events", @@ -54,7 +58,8 @@ func documentHandWritten(api huma.API) { "`decision`, `delegated`, `task_settled` (files " + "lists what the work's code handed back, each a name, mime_type, url and size, uploaded " + "to a persistent conversation's channel and attached to the reply), " + - "`task_cancelled`, `tool_call`, `tool_held`, `tool_approval_ignored`, `tool_updated`, `tool_ran`, `transferred`, `pressed`, `looked_up`, " + + "`task_cancelled`, `tool_call`, `tool_held`, `tool_approval_ignored`, `tool_started`, `tool_updated`, `tool_ran`, " + + "`tool_call.started`, `tool_call.progress`, `tool_call.finished`, `transferred`, `pressed`, `looked_up`, " + "`backchannel`, `interrupted` (turn_id; with synthesis_id, heard_ms, message_start and " + "message_end, the utterance being heard when it was cut, how many milliseconds of it had been, " + "and where all it had been sent is in the reply's message, when any of it had), " + @@ -119,6 +124,11 @@ func documentHandWritten(api huma.API) { "failed and cancelled. tool_started includes tool_call_id, tool, turn_id and started_at, and " + "pre_speech when the tool's connector binding sets one in its policy; " + "tool_ran also includes tool_call_id, and files when the tool was run_code, as on task_settled.\n" + + "`tool_call.started`, `tool_call.progress` and `tool_call.finished` (`ToolCallState`) are a " + + "call's state as a client shows it, whoever runs the call: tool_call_id, tool, turn_id, status " + + "(in_progress, waiting for a person's approval, success, failed or cancelled), progress (the last " + + "`tool_update`, the binding's pre_speech or the approval question), started_at, duration_ms, " + + "approved_at, rejected_at and error. Each carries the whole state: keep the last per tool_call_id.\n" + "A respond command carrying request_id emits command_accepted with a nested command " + "receipt (request_id, user_message_id, assistant_message_id, state, duplicate). Personal " + "persistent text sessions require this ID. A retry with the same text and images returns " + diff --git a/acceleration/internal/api/money.go b/acceleration/internal/api/money.go index 994dd0e266..a30ca0d9d1 100644 --- a/acceleration/internal/api/money.go +++ b/acceleration/internal/api/money.go @@ -88,6 +88,9 @@ func refusedRequest(err error) error { } return answer } + if errors.Is(err, session.ErrToolNotConfigured) { + return toolNotConfigured(err) + } return invalidRequest(err.Error()) } diff --git a/acceleration/internal/api/policies.go b/acceleration/internal/api/policies.go index 3c2a78161e..4c870c8b53 100644 --- a/acceleration/internal/api/policies.go +++ b/acceleration/internal/api/policies.go @@ -42,7 +42,7 @@ type Policy struct { RequireOwnStreamApp *bool `json:"require_own_stream_app,omitempty" doc:"Keep the app out of the router's own Stream app: in app mode it is never written there for want of a registered Stream app of its own, and what it wrote there before can only be read. True at either scope requires it, so an app cannot turn its organization's off. An organization's is set by the router's operator and read here; sending it back unchanged is fine, and changing it is refused."` ResumeHistory *ResumeHistory `json:"resume_history,omitempty" doc:"How the scope's sessions are handed back a conversation they carry on. The most specific level wins, field by field: a server-side session request, then its agent config, then the app's policy, then the organization's, then the router's default (text only, 100 turns, 60000 characters)."` BalanceWarningMicros *int64 `json:"balance_warning_micros,omitempty" minimum:"0" example:"20000000" doc:"The organization's own warning threshold, in millionths of a dollar: at or below it, every answer to a backend says X-Stream-Balance-Warning: low, and the organization's stream event destinations get one account.balance_low. It wins over the tier's. Left out, the tier's applies. Only an organization's policy has one: an app's that sets it is refused."` - ToolOutputRetention *string `json:"tool_output_retention,omitempty" pattern:"^(|none|[1-9][0-9]*[hd])$" patternDescription:"none, a number of hours such as 12h, or a number of days such as 30d" example:"30d" doc:"How long a session keeps what its tools were given and gave back on the turns it records: a tool's arguments, its result, a search's query and the wording of a tool's error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed, so a session's items read in the same order and tool usage still counts the call. Left out, or empty, it is kept as long as the session. The shortest of the organization's, the app's and the agent config's tools.output_retention applies, so an agent can shorten it and never lengthen it. It applies to tool calls recorded after it is set. Deleting a session deletes all of it at once."` + ToolOutputRetention *string `json:"tool_output_retention,omitempty" pattern:"^(|none|[1-9][0-9]*[hd])$" patternDescription:"none, a number of hours such as 12h, or a number of days such as 30d" example:"30d" doc:"How long a session keeps what its tools were given and gave back on the turns it records: a tool's arguments, its result, a search's query and the wording of a tool's error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed, so a session's items read in the same order and tool usage still counts the call. Left out, or empty, it is kept as long as the session. The shortest of the organization's, the app's and the output_retention of the agent config's tool group or connector binding the tool is in applies, so an agent can shorten it and never lengthen it. It applies to tool calls recorded after it is set. Deleting a session deletes all of it at once."` } func (*Policy) TransformSchema(_ huma.Registry, schema *huma.Schema) *huma.Schema { diff --git a/acceleration/internal/api/scheduled_tasks_test.go b/acceleration/internal/api/scheduled_tasks_test.go index aabefc58cb..5ea9ad4602 100644 --- a/acceleration/internal/api/scheduled_tasks_test.go +++ b/acceleration/internal/api/scheduled_tasks_test.go @@ -3016,7 +3016,9 @@ func taskIDs(page ScheduledTaskPage) []string { } func (s *ScheduledTasksSuite) TestACallersOwnScheduleTaskKeepsTheName() { - config := s.agentWithScheduling(nil) + config := s.agentWithScheduling(func(request *AgentConfigRequest) { + request.Tools = &[]ToolGroup{{Name: "mine", Tools: []string{"schedule_task"}}} + }) user := s.data.createUser() s.schedulingScript.plan(llm.ToolCall{ID: store.NewID(), Name: "schedule_task", Arguments: `{"title":"Stretch","instructions":"Remind them to stretch.","in":"PT2H","timezone":"Europe/Amsterdam"}`}) diff --git a/acceleration/internal/api/server.go b/acceleration/internal/api/server.go index a6f1dc4ebc..852dbf6de3 100644 --- a/acceleration/internal/api/server.go +++ b/acceleration/internal/api/server.go @@ -2088,11 +2088,22 @@ func (s *Server) getAgentStats(ctx context.Context, request *getAgentStatsReques return nil, invalidRequest("statistics are not available: no database configured") } + configID := value(request.Agent.ptr()) + if configID != "" { + if _, err := s.configs.AgentConfig(ctx, customerID, configID); err != nil { + return nil, errUnknownConfig + } + } + peak, err := s.store.CustomerConcurrencyPeak(ctx, customerID, request.From, request.To) if err != nil { return nil, err } - overall, tools, err := s.store.CustomerToolUsage(ctx, customerID, request.From, request.To) + turns, err := s.store.CustomerTurns(ctx, customerID, configID, request.From, request.To) + if err != nil { + return nil, err + } + overall, tools, err := s.store.CustomerToolUsage(ctx, customerID, configID, request.From, request.To) if err != nil { return nil, err } @@ -2100,24 +2111,41 @@ func (s *Server) getAgentStats(ctx context.Context, request *getAgentStatsReques stats := AgentStats{ PeakConcurrentSessions: peak.Sessions, PeakAt: peak.At, - ToolCalls: toolStatsOf(overall), + Sessions: turns.Sessions, + AvgTurns: share(turns.Turns, turns.Sessions), + ToolCalls: toolStatsOf(overall, turns.Sessions), Tools: make([]ToolStats, 0, len(tools)), } for _, tool := range tools { - stats.Tools = append(stats.Tools, toolStatsOf(tool)) + stats.Tools = append(stats.Tools, toolStatsOf(tool, turns.Sessions)) } return &getAgentStatsResponse{Body: stats}, nil } -func toolStatsOf(usage store.ToolUsage) ToolStats { +func toolStatsOf(usage store.ToolUsage, sessions int64) ToolStats { return ToolStats{ - Tool: usage.Tool, - Calls: usage.Calls, - Failed: usage.Failed, - AvgTokens: usage.AvgTokens, - AvgMs: usage.AvgMs, - P99Ms: usage.P99Ms, + Tool: usage.Tool, + Calls: usage.Calls, + Failed: usage.Failed, + Sessions: usage.Sessions, + SessionShare: share(usage.Sessions, sessions), + AvgTurnsBeforeUse: usage.AvgTurnsBeforeUse, + AvgTokens: usage.AvgTokens, + AvgMs: usage.AvgMs, + P99Ms: usage.P99Ms, + Loading: usage.Loading, + DefinitionTokens: usage.DefinitionTokens, + SummaryTokens: usage.SummaryTokens, + } +} + +// share is part over whole, nil when whole is none. +func share(part, whole int64) *float64 { + if whole == 0 { + return nil } + ratio := float64(part) / float64(whole) + return &ratio } // runRollup aggregates every customer's request rows into a rollup table. It is a @@ -2410,13 +2438,16 @@ func (s *Server) registerServer(api huma.API) { Path: "/v1/stats/agents", Summary: "How the calling customer's agents ran, across all of them", Description: "The most agents that were running at once, and which tools they called: how often, " + - "how much each call put in front of the model and how long it took.\n" + + "in how many sessions and after how many turns, how much each call put in front of the model, " + + "what the tool's definition costs every turn that carries it and how long a call took.\n" + + "With agent, the sessions, turns and tools are that agent config's alone; the peak is still " + + "every agent's.\n" + "One figure for the whole window rather than a row per bucket, because neither a peak nor " + "a percentile can be summed out of smaller ones.", Responses: map[string]*huma.Response{ "200": {Description: "The window's figures"}, }, - Errors: []int{http.StatusBadRequest, http.StatusUnauthorized, http.StatusForbidden}, + Errors: []int{http.StatusBadRequest, http.StatusUnauthorized, http.StatusForbidden, http.StatusNotFound}, }, s.getAgentStats) huma.Register(api, huma.Operation{ OperationID: "runRollup", @@ -2552,8 +2583,9 @@ type getActivityResponse struct { } type getAgentStatsRequest struct { - From time.Time `query:"from" doc:"Start of the window, inclusive." required:"true"` - To time.Time `query:"to" doc:"End of the window, exclusive." required:"true"` + From time.Time `query:"from" doc:"Start of the window, inclusive." required:"true"` + To time.Time `query:"to" doc:"End of the window, exclusive." required:"true"` + Agent optionalParam[string] `query:"agent" doc:"The id of an agent config: only its sessions are counted. Omitted is every agent."` } type getAgentStatsResponse struct { @@ -2564,18 +2596,26 @@ type getAgentStatsResponse struct { type AgentStats struct { PeakConcurrentSessions int64 `json:"peak_concurrent_sessions" doc:"The most sessions open at the same time, each one an agent running. A session still open counts up to now."` PeakAt *time.Time `json:"peak_at" nullable:"true" doc:"When that peak was first reached. Null when no session was open in the window."` + Sessions int64 `json:"sessions" doc:"Sessions that took a turn in the window: what a tool's session_share is out of."` + AvgTurns *float64 `json:"avg_turns" nullable:"true" doc:"Turns a session took in the window, on average. A turn is a reply to the person, not a rewound one or a delegated task's. Null when no session took one."` ToolCalls ToolStats `json:"tool_calls" doc:"Every tool call in the window, as one. Its tool is empty."` Tools []ToolStats `json:"tools" nullable:"false" doc:"One row per tool, most called first."` } // ToolStats is how often a tool ran and what it cost. type ToolStats struct { - Tool string `json:"tool"` - Calls int64 `json:"calls"` - Failed int64 `json:"failed" doc:"Calls that ended in an error."` - AvgTokens float64 `json:"avg_tokens" doc:"What a call put in front of the model, its arguments and its result, estimated at four characters a token."` - AvgMs *float64 `json:"avg_ms" nullable:"true" doc:"From the call starting to its result. Null when none was timed."` - P99Ms *float64 `json:"p99_ms" nullable:"true"` + Tool string `json:"tool"` + Calls int64 `json:"calls"` + Failed int64 `json:"failed" doc:"Calls that ended in an error."` + Sessions int64 `json:"sessions" doc:"Sessions that called it at least once."` + SessionShare *float64 `json:"session_share" nullable:"true" minimum:"0" doc:"Sessions that called it, out of the window's sessions, from 0 to 1. Null when no session took a turn."` + AvgTurnsBeforeUse *float64 `json:"avg_turns_before_use" nullable:"true" doc:"Turns a session took before its first call, on average: 0 is a call on the first turn. Null on the row over every tool."` + AvgTokens float64 `json:"avg_tokens" doc:"What a call put in front of the model, its arguments and its result, estimated at four characters a token."` + AvgMs *float64 `json:"avg_ms" nullable:"true" doc:"From the call starting to its result. Null when none was timed."` + P99Ms *float64 `json:"p99_ms" nullable:"true"` + Loading string `json:"loading" doc:"How the latest call had the tool offered: full, summary or deferred. Empty when no call recorded it, as calls made before it was recorded and built-in tools."` + DefinitionTokens *int64 `json:"definition_tokens" nullable:"true" doc:"What the tool's whole definition, its name, description and schema, costs every turn that carries it, estimated at four characters a token. Null when no call recorded it."` + SummaryTokens *int64 `json:"summary_tokens" nullable:"true" doc:"What its definition costs offered as a summary. Null unless its loading is summary."` } type runRollupRequest struct { diff --git a/acceleration/internal/api/server_test.go b/acceleration/internal/api/server_test.go index acc3ce5d26..5f4ddf736b 100644 --- a/acceleration/internal/api/server_test.go +++ b/acceleration/internal/api/server_test.go @@ -350,6 +350,17 @@ func (s *ServerSuite) TestAgentStatsReportThePeakAndTheToolsCalled() { s.Equal("lookup_order", stats.Tools[0].Tool) s.Require().NotNil(stats.Tools[0].AvgMs) s.InDelta(250, *stats.Tools[0].AvgMs, 0.01) + s.EqualValues(1, stats.Sessions) + s.Require().NotNil(stats.Tools[0].SessionShare) + s.InDelta(1, *stats.Tools[0].SessionShare, 0.001, "the one session that took a turn called it") + s.Require().NotNil(stats.Tools[0].AvgTurnsBeforeUse) + s.Zero(*stats.Tools[0].AvgTurnsBeforeUse, "it was called on the first turn") +} + +func (s *ServerSuite) TestAgentStatsOfAnAgentThatIsNotTheCallersAreNotFound() { + var failure map[string]any + s.Equal(http.StatusNotFound, s.serverClient.do(http.MethodGet, + "/v1/stats/agents?from="+s.from()+"&to="+s.to(24*time.Hour)+"&agent="+s.utils.uuid(), nil, &failure)) } func (s *ServerSuite) TestProvidersReportLiveHealth() { diff --git a/acceleration/internal/api/session_tools.go b/acceleration/internal/api/session_tools.go index f3c3f45f41..ac98f72e8c 100644 --- a/acceleration/internal/api/session_tools.go +++ b/acceleration/internal/api/session_tools.go @@ -76,6 +76,7 @@ type OfferedTool struct { Parameters map[string]any `json:"parameters,omitempty" doc:"The JSON Schema of its arguments."` Tokens int64 `json:"tokens" doc:"Roughly what offering it costs on every request, in tokens: its name, description and schema at four characters a token."` Source string `json:"source,omitempty" enum:"session,worker,connector,builtin" doc:"Where the tool came from: session, declared by whoever opened the session; worker, hosted by a worker connected when the session opened; connector, opened by one of the agent's connector bindings; builtin, the router's own. A session's tools are fixed when it opens, so a worker that was not connected then leaves none of its tools here. Absent for a session kept before sources were recorded."` + Loading *ToolLoading `json:"loading,omitempty" doc:"How the tool is offered, as its tool group or connector binding says: full for the router's own. A deferred tool is listed once the model has loaded it. Absent for a session that has ended."` } func (*OfferedTool) TransformSchema(_ huma.Registry, schema *huma.Schema) *huma.Schema { @@ -190,6 +191,8 @@ func toolsChanged(err error) error { return conflict(err.Error()) case errors.Is(err, session.ErrNoCallerTool): return errNoCallerTool + case errors.Is(err, session.ErrToolNotConfigured): + return toolNotConfigured(err) case errors.Is(err, sts.ErrToolsFixed): return errToolsFixed case errors.Is(err, agent.ErrSpeechEnded): @@ -209,10 +212,12 @@ func (s *Server) listSessionTools(ctx context.Context, request *listSessionTools } var tools []llm.Tool sources := map[string]string{} + loadings := map[string]ToolLoading{} if found.Live != nil { tools = found.Live.ToolDefinitions() for _, tool := range tools { sources[tool.Name] = string(found.Live.ToolSource(tool.Name)) + loadings[tool.Name] = ToolLoading(found.Live.ToolLoading(tool.Name)) } } else { stored, err := s.store.SessionTools(ctx, request.Id) @@ -231,10 +236,14 @@ func (s *Server) listSessionTools(ctx context.Context, request *listSessionTools offered := OfferedTools{Tools: []OfferedTool{}} for _, tool := range tools { tokens := llm.ToolTokens(tool) - offered.Tools = append(offered.Tools, OfferedTool{ + one := OfferedTool{ Name: tool.Name, Description: tool.Description, Parameters: tool.Parameters, Tokens: tokens, Source: sources[tool.Name], - }) + } + if loading, known := loadings[tool.Name]; known { + one.Loading = &loading + } + offered.Tools = append(offered.Tools, one) offered.Tokens += tokens } return &listSessionToolsResponse{Body: offered}, nil diff --git a/acceleration/internal/api/sessionws.go b/acceleration/internal/api/sessionws.go index f7d7cc9bc7..4937d38bdc 100644 --- a/acceleration/internal/api/sessionws.go +++ b/acceleration/internal/api/sessionws.go @@ -11,6 +11,7 @@ import ( "strings" "time" + "github.com/danielgtaylor/huma/v2" "github.com/gorilla/websocket" "github.com/GetStream/Vision-Agents/acceleration/internal/agent" @@ -625,6 +626,9 @@ func frameOf(event session.Event) (frame, bool) { } return ignored, true + case session.ToolCallChanged: + return toolStateFrame(typed), true + case agent.ToolUpdated: return frame{ "type": "tool_updated", "id": typed.ID, "turn_id": typed.TurnID, "tool": typed.Tool, @@ -999,6 +1003,75 @@ func withLine(rendered frame, line string) frame { return rendered } +// toolStateFrame renders one change to a tool call's state, as ToolCallState describes it. +func toolStateFrame(changed session.ToolCallChanged) frame { + state := changed.State + rendered := frame{ + "type": "tool_call." + string(changed.Phase), + "tool_call_id": state.ID, + "tool": state.Tool, + "turn_id": state.TurnID, + "status": string(state.Status), + "started_at": state.StartedAt, + "duration_ms": state.Duration.Milliseconds(), + } + for key, value := range map[string]string{"progress": state.Progress, "error": state.Error} { + if value != "" { + rendered[key] = value + } + } + for key, at := range map[string]time.Time{"approved_at": state.ApprovedAt, "rejected_at": state.RejectedAt} { + if !at.IsZero() { + rendered[key] = at + } + } + return rendered +} + +// ToolCallState is one change to a tool call's state, sent to every watcher of the session. +type ToolCallState struct { + Type ToolCallFrameType `json:"type"` + ToolCallID string `json:"tool_call_id" doc:"The call, as tool_call, tool_started and tool_ran name it."` + Tool string `json:"tool"` + TurnID string `json:"turn_id"` + Status ToolCallStatus `json:"status"` + Progress string `json:"progress,omitempty" doc:"The last thing said about the call while it runs, such as \"Searching for flights\": a tool_update, the connector binding's pre_speech, or the question it waits on."` + StartedAt time.Time `json:"started_at"` + DurationMS int64 `json:"duration_ms" doc:"From started_at to this change; on tool_call.finished, how long the call took."` + ApprovedAt *time.Time `json:"approved_at,omitempty" doc:"When a person allowed the call."` + RejectedAt *time.Time `json:"rejected_at,omitempty" doc:"When a person declined the call, or its wait for them passed."` + Error string `json:"error,omitempty" doc:"Why a failed call failed, or what a declined one shows, such as \"Not allowed in time\"."` +} + +func (*ToolCallState) TransformSchema(_ huma.Registry, schema *huma.Schema) *huma.Schema { + schema.Description = "One change to a tool call's state. Every watcher is sent tool_call.started once, " + + "tool_call.progress for each change while it runs or waits, and tool_call.finished once, carrying " + + "the call's whole state each time, so a client keeps the last one it got per tool_call_id." + return schema +} + +// ToolCallFrameType is which of a call's changes a ToolCallState is. +type ToolCallFrameType string + +func (ToolCallFrameType) Schema(registry huma.Registry) *huma.Schema { + return namedEnum(registry, "ToolCallFrameType", + "tool_call.started is the call's first frame, tool_call.progress one while it runs or waits, "+ + "and tool_call.finished its last.", + "tool_call.started", "tool_call.progress", "tool_call.finished") +} + +// ToolCallStatus is where a tool call stands. +type ToolCallStatus string + +func (ToolCallStatus) Schema(registry huma.Registry) *huma.Schema { + return namedEnum(registry, "ToolCallStatus", + "in_progress while it runs, waiting while it is held for a person's approval, and success, "+ + "failed or cancelled once it finished. A call declined, or whose approval nobody gave in "+ + "time, is cancelled with rejected_at.", + string(session.ToolCallInProgress), string(session.ToolCallWaiting), string(session.ToolCallSuccess), + string(session.ToolCallFailed), string(session.ToolCallCancelled)) +} + func participantOf(participant stt.Participant) frame { return frame{ "id": participant.ID, diff --git a/acceleration/internal/api/sessionws_frames_test.go b/acceleration/internal/api/sessionws_frames_test.go index 35d06aac44..bed6196727 100644 --- a/acceleration/internal/api/sessionws_frames_test.go +++ b/acceleration/internal/api/sessionws_frames_test.go @@ -58,6 +58,22 @@ func TestAToolStartedWithoutPreSpeechIsTheFrameItAlwaysWas(t *testing.T) { string(raw)) } +func TestAFinishedToolCallCarriesItsWholeState(t *testing.T) { + at := time.Date(2026, 10, 7, 12, 0, 0, 0, time.UTC) + + sent, ok := frameOf(session.ToolCallChanged{Phase: session.ToolCallFinished, State: session.ToolCallState{ + ID: "call-1", Tool: "book_flight", TurnID: "turn-1", Status: session.ToolCallSuccess, + Progress: "Booking", StartedAt: at, Duration: 1500 * time.Millisecond, ApprovedAt: at.Add(time.Second), + }}) + + require.True(t, ok) + raw, err := json.Marshal(sent) + require.NoError(t, err) + require.JSONEq(t, `{"type":"tool_call.finished","tool_call_id":"call-1","tool":"book_flight","turn_id":"turn-1", + "status":"success","progress":"Booking","started_at":"2026-10-07T12:00:00Z","duration_ms":1500, + "approved_at":"2026-10-07T12:00:01Z"}`, string(raw)) +} + func TestAToolStartedCarriesItsBindingsPreSpeech(t *testing.T) { at := time.Date(2026, 10, 7, 12, 0, 0, 0, time.UTC) diff --git a/acceleration/internal/api/sync.go b/acceleration/internal/api/sync.go index 05de1c54d5..5aa8afe9c5 100644 --- a/acceleration/internal/api/sync.go +++ b/acceleration/internal/api/sync.go @@ -59,7 +59,7 @@ type SyncAgentRequest struct { SandboxOptions *SandboxOptions `json:"sandbox_options,omitempty"` Tags *map[string]string `json:"tags,omitempty"` Agents *[]ChildAgent `json:"agents,omitempty" maxItems:"16" doc:"agent.yaml's agents: the app's other text agents this one may hand work to, each as a tool, by name. Sent, they replace the stored list whole, an empty list removing it; left out, the stored list stays. An agent named must already exist, so a directory is synced after the agents it names."` - Tools *AgentTools `json:"tools,omitempty" doc:"How connector tools are offered. A setting left out keeps what is stored."` + Tools *[]ToolGroup `json:"tools,omitempty" maxItems:"64" doc:"agent.yaml's tools: the functions an SDK provides that the agent may offer, in named groups, as AgentConfigRequest's tools names them, with the same rules. Sent, they replace the stored groups whole, an empty list removing them. Left out, the stored ones stay."` Calls *AgentCalls `json:"calls,omitempty" doc:"The tools an agent on a phone call is offered: end the call, press keys, transfer. A tool left out keeps what is stored; a tool sent replaces its settings whole."` Scheduling *AgentScheduling `json:"scheduling,omitempty" doc:"Whether the agent may schedule tasks for its users, and the defaults those tasks take. A setting left out keeps what is stored."` ResumeHistory *ResumeHistory `json:"resume_history,omitempty" doc:"How a conversation is handed back to the model when a session carries it on. A setting left out keeps what is stored."` @@ -282,7 +282,7 @@ func (s *Server) syncAgent(ctx context.Context, request *syncAgentRequest) (*syn if message, ok := teamTagComplaint(config.Tags, existing.Tags); !ok { return nil, invalidRequest(message) } - if message, ok := outputRetentionComplaint(config); !ok { + if message, ok := toolsComplaint(config); !ok { return nil, invalidRequest(message) } if message, ok, err := s.sandboxBindingsComplaint(ctx, customerID, config); err != nil { @@ -603,8 +603,9 @@ func applySettings(config *store.AgentConfig, body SyncAgentRequest) { config.Connectors = storedBindings(*body.Connectors) config.LeftOutConnectors = leftOutOf(body.LeftOutConnectors) } - config.ProgressiveTools = progressiveOf(config.ProgressiveTools, body.Tools) - config.ToolOutputRetention = outputRetentionOf(config.ToolOutputRetention, body.Tools) + if body.Tools != nil { + config.ToolGroups = toolGroupsOf(*body.Tools) + } config.Scheduling = schedulingOr(config.Scheduling, body.Scheduling) config.Calls = callsOr(config.Calls, body.Calls) config.ResumeHistory = resumeHistoryOr(config.ResumeHistory, body.ResumeHistory) diff --git a/acceleration/internal/api/sync_test.go b/acceleration/internal/api/sync_test.go index f154a4d51d..b5e8417914 100644 --- a/acceleration/internal/api/sync_test.go +++ b/acceleration/internal/api/sync_test.go @@ -118,12 +118,24 @@ func (s *SyncSuite) TestASyncNamingTheAgentTypeOrATypeTwiceIsRefused() { s.Empty(s.configsNamed("rota")) } -func (s *SyncSuite) TestSyncingAnAgentStoresWhetherItsToolsAreOfferedProgressively() { - result := s.sync(map[string]any{"name": "concierge", "hash": "v1", "tools": map[string]any{"progressive": true}}) - s.True(value(result.Config.Tools.Progressive)) +func (s *SyncSuite) TestSyncingAnAgentStoresItsToolGroups() { + result := s.sync(map[string]any{"name": "concierge", "hash": "v1", "tools": []any{ + map[string]any{"name": "booking", "loading": "summary", "tools": []any{"lookup_booking"}}, + }}) + s.Require().NotNil(result.Config.Tools) + s.Equal(ToolLoadingSummary, value((*result.Config.Tools)[0].Loading)) again := s.sync(map[string]any{"name": "concierge", "hash": "v2"}) - s.True(value(again.Config.Tools.Progressive), "a directory that says nothing leaves what is stored") + s.Require().NotNil(again.Config.Tools, "a directory that says nothing leaves what is stored") + s.Equal([]string{"lookup_booking"}, (*again.Config.Tools)[0].Tools) +} + +func (s *SyncSuite) TestSyncingTheToolSettingsOfBeforeGroupsIsRefused() { + status, _ := s.serverClient.failure(http.MethodPost, "/v1/agents/sync", + map[string]any{"name": "concierge", "hash": "v1", "tools": map[string]any{"progressive": true}}) + + s.Equal(http.StatusBadRequest, status) + s.Empty(s.configsNamed("concierge")) } func (s *SyncSuite) TestSyncingAnAgentStoresItsScheduling() { @@ -923,16 +935,18 @@ func (s *SyncSuite) TestTwoSyncedSandboxBindingsGivingTheSandboxOneEnvVarAreRefu s.Empty(s.configsNamed("builder")) } -func (s *SyncSuite) TestSyncingAnAgentStoresHowLongItKeepsToolOutput() { - result := s.sync(map[string]any{"name": "concierge", "hash": "v1", "tools": map[string]any{"output_retention": "30d"}}) - s.Equal("30d", value(result.Config.Tools.OutputRetention)) - - again := s.sync(map[string]any{"name": "concierge", "hash": "v2"}) - s.Equal("30d", value(again.Config.Tools.OutputRetention), "a directory that says nothing leaves what is stored") +func (s *SyncSuite) TestSyncingAnAgentStoresHowLongEachGroupKeepsToolOutput() { + result := s.sync(map[string]any{"name": "concierge", "hash": "v1", "tools": []any{ + map[string]any{"name": "cards", "output_retention": "30d", "tools": []any{"charge_card"}}, + }}) + s.Require().NotNil(result.Config.Tools) + s.Equal("30d", value((*result.Config.Tools)[0].OutputRetention)) status, failure := s.serverClient.failure(http.MethodPost, "/v1/agents/sync", map[string]any{ - "name": "concierge", "hash": "v3", "tools": map[string]any{"output_retention": "3651d"}, + "name": "concierge", "hash": "v3", "tools": []any{ + map[string]any{"name": "cards", "output_retention": "3651d", "tools": []any{"charge_card"}}, + }, }) s.Equal(http.StatusBadRequest, status) - s.Contains(failure, "tools.output_retention") + s.Contains(failure, "tools[0].output_retention") } diff --git a/acceleration/internal/api/tool_output_retention_test.go b/acceleration/internal/api/tool_output_retention_test.go index 621a66aa96..adab5efae1 100644 --- a/acceleration/internal/api/tool_output_retention_test.go +++ b/acceleration/internal/api/tool_output_retention_test.go @@ -28,7 +28,7 @@ func (s *ToolOutputRetentionSuite) TestAnAppPolicyOfNoneRecordsNoneOfAToolsOutpu map[string]any{"tool_output_retention": "none"}, nil)) // The config would keep it a week. The app's policy is the longest any of its agents keeps. config := s.createConfig(map[string]any{"name": "orders", "llm": "tooling/tool-model", - "tools": map[string]any{"output_retention": "7d"}}) + "tools": []any{map[string]any{"name": "orders", "output_retention": "7d", "tools": []any{lookupOrder}}}}) result := s.ranTheTool(config.Id, map[string]any{"output": "it ships tomorrow"}) @@ -36,6 +36,30 @@ func (s *ToolOutputRetentionSuite) TestAnAppPolicyOfNoneRecordsNoneOfAToolsOutpu s.Equal(map[string]any{"call_id": result.callID, "redacted": true}, value(result.Payload)) } +func (s *ToolOutputRetentionSuite) TestAGroupThatKeepsNoneRecordsNoneOfItsToolsOutput() { + config := s.createConfig(map[string]any{"name": "orders", "llm": "tooling/tool-model", + "tools": []any{map[string]any{"name": "orders", "output_retention": "none", "tools": []any{lookupOrder}}}}) + + result := s.ranTheTool(config.Id, map[string]any{"output": "it ships tomorrow"}) + + s.Empty(value(result.Text)) + s.Equal(map[string]any{"call_id": result.callID, "redacted": true}, value(result.Payload)) +} + +func (s *ToolOutputRetentionSuite) TestASessionDeclaringAToolNoGroupNamesIsRefused() { + config := s.createConfig(map[string]any{"name": "orders", "llm": "tooling/tool-model", "tools": []any{ + map[string]any{"name": "orders", "tools": []any{lookupOrder}}, + }}) + + status, failure := s.serverClient.failure(http.MethodPost, "/v1/agents/sessions", map[string]any{ + "config_id": config.Id, + "tools": []any{map[string]any{"name": "cancel_order", "description": "cancel an order"}}, + }) + + s.Equal(http.StatusBadRequest, status) + s.Contains(failure, "no tool group of the agent's config names the tool: cancel_order") +} + func (s *ToolOutputRetentionSuite) TestWithoutARetentionAToolsOutputIsRecordedWhole() { config := s.createConfig(map[string]any{"name": "orders", "llm": "tooling/tool-model"}) @@ -104,7 +128,12 @@ func (s *ToolOutputRetentionSuite) ranTheTool(configID string, answer map[string return recordedResult{AgentResponseItem: *found, callID: callID} } +// createConfig creates the config body describes, naming the caller's own tool in a group +// when body names none. func (s *ToolOutputRetentionSuite) createConfig(body map[string]any) AgentConfig { + if _, named := body["tools"]; !named { + body["tools"] = []any{map[string]any{"name": "orders", "tools": []any{lookupOrder}}} + } var created AgentConfig s.Require().Equal(http.StatusCreated, s.serverClient.do(http.MethodPost, "/v1/agents/configs", body, &created)) diff --git a/acceleration/internal/dispatch/tools.go b/acceleration/internal/dispatch/tools.go index 910a885f53..ded603f7d5 100644 --- a/acceleration/internal/dispatch/tools.go +++ b/acceleration/internal/dispatch/tools.go @@ -172,13 +172,14 @@ func (w *Worker) progressOf(id string) listening { return w.progress[id] } -// Host records that a worker runs these tools for sessions under an agent id, replacing -// whatever it offered for that agent before. The offer is scoped to the worker's own -// customer: this is the pool, not the gate. +// EveryAgent is the agent id of tools a worker hosts for every agent of its customer. A +// session is offered one of them only when its config's tool groups name it. +const EveryAgent = "" + +// Host records that a worker runs these tools for sessions under an agent id, or EveryAgent, +// replacing whatever it offered for that agent before. The offer is scoped to the worker's +// own customer: this is the pool, not the gate. func (p *Pool) Host(worker *Worker, agentID string, tools []Tool, timeout time.Duration) error { - if agentID == "" { - return errors.New("dispatch: hosted tools need the agent they are for") - } if len(tools) == 0 { return errors.New("dispatch: hosting no tools is not hosting") } @@ -252,6 +253,16 @@ func (p *Pool) AgentsHostedBy(worker *Worker) []string { return agents } +// NoWorkerError is a hosted call no connected worker runs the tool for: the worker that +// hosted it when the session opened has gone, or every one running it is full. +type NoWorkerError struct { + Tool string +} + +func (e NoWorkerError) Error() string { + return fmt.Sprintf("dispatch: no worker is running %s now", e.Tool) +} + // RunHosted sends one call to a worker hosting that tool for the agent and waits for its // answer, the worker's own timeout, or the caller giving up. // @@ -284,7 +295,7 @@ func (p *Pool) RunHosted(ctx context.Context, customerID, agentID string, call T } if chosen == nil { p.mu.Unlock() - return nil, fmt.Errorf("dispatch: no worker is running %s now", call.Name) + return nil, NoWorkerError{Tool: call.Name} } chosen.worker.mu.Lock() if _, duplicate := chosen.worker.results[call.ID]; duplicate { diff --git a/acceleration/internal/harness/AGENTS.md b/acceleration/internal/harness/AGENTS.md index 33101b8a55..4e4e6e788f 100644 --- a/acceleration/internal/harness/AGENTS.md +++ b/acceleration/internal/harness/AGENTS.md @@ -4,6 +4,6 @@ Decides what the reply model is sent. Providers cache a prompt by its prefix, an - The system prompt (`prompt()` → instructions) is the same on every turn of a call. Never put anything that changes per turn into it. - What changes goes after the conversation: `Turn.Tell` for something true from now on (kept, `Remember` returns it), `Turn.Note` for this reply only (not kept). Delegated results go as a `{"harness":…}` envelope followed by a fixed-wording note (`note()`); never interpolate colleague, tool or caller text into a note's words. -- History is append-only. Don't rewrite, reorder or drop earlier messages; tools stay offered and are switched off with `tool_choice`. One exception: a model whose `ToolChoices` lack `none` (Muse, Ollama) gets no tools on that reply instead. +- History is append-only. Don't rewrite, reorder or drop earlier messages; tools stay offered and are switched off with `tool_choice`. One exception: a model whose `ToolChoices` lack `none` (Muse, Ollama) gets no tools on that reply instead. Another: a deferred tool (`Tools.Deferred`) the model loads with `load_tool` is added after every tool offered before it (`agent/deferred_tools.go`), which costs the provider's cache once per load and is the price of not sending every schema on every turn. Guarded by `TestInstructionsAreTheSameWhateverTheTurnCarries` and `TestAReplyCarryingAnAnswerContinuesTheStoredResponse` (`prefix_test.go`). See `.factory/features/harness.md`, "The prompt every turn shares". diff --git a/acceleration/internal/harness/deferred.go b/acceleration/internal/harness/deferred.go new file mode 100644 index 0000000000..f49a38f68b --- /dev/null +++ b/acceleration/internal/harness/deferred.go @@ -0,0 +1,165 @@ +package harness + +import ( + "encoding/json" + "slices" + "strconv" + "strings" + "unicode" +) + +// SearchToolName and LoadToolName are how a model finds and loads the tools it was not +// offered at the start (Tools.Deferred). No other tool may take either name. +const ( + SearchToolName = "search_tool" + LoadToolName = "load_tool" +) + +// DeferredToolNames are the names the router keeps for loading deferred tools. +var DeferredToolNames = []string{SearchToolName, LoadToolName} + +// searchLimit is the most tools one search_tool call returns, enough to choose between +// without spending the context deferring them saved. +const searchLimit = 10 + +// summaryLimit caps the one line a tool is summarised by, in runes. +const summaryLimit = 200 + +// SearchTool is search_tool as the model is offered it. +func SearchTool() Tool { + return Tool{ + Name: SearchToolName, + Description: "Find tools you were not offered yet by what they do. Returns up to 10 tools, each " + + "with its name and a line on what it does; load the ones you need with load_tool before " + + "calling them. Search whenever none of your tools does what the conversation needs.", + Parameters: map[string]any{ + "type": "object", + "required": []string{"query"}, + "properties": map[string]any{ + "query": map[string]any{"type": "string", "description": "Words for what the tool should do, such as look up a booking."}, + }, + }, + } +} + +// LoadTool is load_tool as the model is offered it. +func LoadTool() Tool { + return Tool{ + Name: LoadToolName, + Description: "Load tools search_tool found, by name. Returns each one's full description and " + + "input schema, and you can call them from your next step on.", + Parameters: map[string]any{ + "type": "object", + "required": []string{"names"}, + "properties": map[string]any{ + "names": map[string]any{"type": "array", "items": map[string]any{"type": "string"}, "minItems": 1}, + }, + }, + } +} + +// SearchDeferred is what search_tool answers query with: the deferred tools whose name or +// description shares a word with it, those sharing the most first and a word in the name +// counting for more, in the order they were declared when they tie. +func SearchDeferred(deferred []Tool, query string) string { + words := wordsOf(query) + type match struct { + tool Tool + score int + } + var matches []match + for _, tool := range deferred { + named, described := wordsOf(tool.Name), wordsOf(tool.Description) + score := 0 + for _, word := range words { + if slices.Contains(named, word) { + score += 3 + } + if slices.Contains(described, word) { + score++ + } + } + if score > 0 { + matches = append(matches, match{tool: tool, score: score}) + } + } + slices.SortStableFunc(matches, func(a, b match) int { return b.score - a.score }) + if len(matches) == 0 { + return "No tool matches " + strconv.Quote(query) + ". Try other words for what it should do." + } + found := make([]map[string]string, 0, min(len(matches), searchLimit)) + for _, match := range matches[:min(len(matches), searchLimit)] { + found = append(found, map[string]string{"name": match.tool.Name, "description": Summary(match.tool.Description)}) + } + encoded, _ := json.Marshal(map[string]any{"tools": found}) + return string(encoded) +} + +// LoadDeferred is what load_tool answers with for the deferred tools called names: each one's +// full description and input schema, and the names that are no deferred tool. found are the +// names it loaded, in the order asked. +func LoadDeferred(deferred []Tool, names []string) (result string, found []string) { + var loaded []map[string]any + var unknown []string + for _, name := range names { + name = strings.TrimSpace(name) + at := slices.IndexFunc(deferred, func(tool Tool) bool { return tool.Name == name }) + if at < 0 { + unknown = append(unknown, name) + continue + } + if slices.Contains(found, name) { + continue + } + found = append(found, name) + loaded = append(loaded, map[string]any{ + "name": name, "description": deferred[at].Description, "input_schema": deferred[at].Parameters, + }) + } + answer := map[string]any{"loaded": loaded} + if len(unknown) > 0 { + answer["unknown"] = unknown + answer["note"] = "Find a tool's name with search_tool before loading it." + } + encoded, _ := json.Marshal(answer) + return string(encoded), found +} + +// LoadedNames are the names a load_tool call's arguments ask for, or none when they cannot +// be read. +func LoadedNames(arguments string) []string { + var asked struct { + Names []string `json:"names"` + } + if err := json.Unmarshal([]byte(arguments), &asked); err != nil { + return nil + } + return asked.Names +} + +// Summary is the first line of a description, cut at a word to 200 runes. +func Summary(description string) string { + var first string + for line := range strings.Lines(description) { + if first = strings.TrimSpace(line); first != "" { + break + } + } + runes := []rune(first) + if len(runes) <= summaryLimit { + return first + } + cut := string(runes[:summaryLimit]) + if space := strings.LastIndex(cut, " "); space > 0 { + cut = cut[:space] + } + return cut + "…" +} + +// wordsOf is text as lowercase words, split at anything but a letter or a digit, so the +// tool lookup_booking is found by "booking". +func wordsOf(text string) []string { + return strings.FieldsFunc(strings.ToLower(text), func(r rune) bool { + return !unicode.IsLetter(r) && !unicode.IsDigit(r) + }) +} diff --git a/acceleration/internal/harness/deferred_test.go b/acceleration/internal/harness/deferred_test.go new file mode 100644 index 0000000000..733d6db338 --- /dev/null +++ b/acceleration/internal/harness/deferred_test.go @@ -0,0 +1,90 @@ +package harness + +import ( + "encoding/json" + "strings" + "testing" + + "github.com/stretchr/testify/suite" +) + +// DeferredToolsSuite covers how the model finds and loads the tools held back from it. +type DeferredToolsSuite struct { + suite.Suite + deferred []Tool +} + +func TestDeferredToolsSuite(t *testing.T) { + suite.Run(t, new(DeferredToolsSuite)) +} + +func (s *DeferredToolsSuite) SetupTest() { + s.deferred = []Tool{ + {Name: "lookup_booking", Description: "Find a booking by its reference.\n\nThe reference is six letters."}, + {Name: "refund_booking", Description: "Give the traveller their money back for a cancelled trip."}, + {Name: "void_ticket", Description: "Void a ticket issued today.", Parameters: map[string]any{"type": "object"}}, + } +} + +// found are the names a search_tool answer lists, in the order it lists them. +func (s *DeferredToolsSuite) found(answer string) []string { + var listed struct { + Tools []struct{ Name, Description string } `json:"tools"` + } + s.Require().NoError(json.Unmarshal([]byte(answer), &listed), answer) + var names []string + for _, tool := range listed.Tools { + names = append(names, tool.Name) + } + return names +} + +func (s *DeferredToolsSuite) TestASearchFindsAToolByAWordOfItsDescription() { + for _, query := range []string{"money back", "the MONEY", "trip got cancelled, money?"} { + found := s.found(SearchDeferred(s.deferred, query)) + s.Require().NotEmpty(found, query) + s.Equal("refund_booking", found[0], query) + } +} + +func (s *DeferredToolsSuite) TestASearchRanksAToolNamedByTheWordFirst() { + found := s.found(SearchDeferred(s.deferred, "booking")) + + s.Equal([]string{"lookup_booking", "refund_booking"}, found) +} + +func (s *DeferredToolsSuite) TestASearchListsEachToolByItsFirstLine() { + answer := SearchDeferred(s.deferred, "reference") + + s.Contains(answer, "Find a booking by its reference.") + s.NotContains(answer, "six letters") +} + +func (s *DeferredToolsSuite) TestASearchThatMatchesNothingSaysSo() { + answer := SearchDeferred(s.deferred, "weather") + + s.True(strings.HasPrefix(answer, `No tool matches "weather".`), answer) +} + +func (s *DeferredToolsSuite) TestALoadHandsOverTheWholeToolAndNamesWhatItCouldNotFind() { + answer, found := LoadDeferred(s.deferred, []string{"void_ticket", "book_hotel", "void_ticket"}) + + s.Equal([]string{"void_ticket"}, found) + var loaded struct { + Loaded []struct { + Name string `json:"name"` + InputSchema map[string]any `json:"input_schema"` + } `json:"loaded"` + Unknown []string `json:"unknown"` + } + s.Require().NoError(json.Unmarshal([]byte(answer), &loaded)) + s.Len(loaded.Loaded, 1) + s.Equal(map[string]any{"type": "object"}, loaded.Loaded[0].InputSchema) + s.Equal([]string{"book_hotel"}, loaded.Unknown) +} + +func (s *DeferredToolsSuite) TestAToolBothOfferedAndDeferredIsRefused() { + tools := Tools{Tools: s.deferred[:1], Deferred: s.deferred} + + s.ErrorContains(tools.Validate(), "lookup_booking is declared twice") +} diff --git a/acceleration/internal/harness/tools.go b/acceleration/internal/harness/tools.go index 41d74aaa32..3c81004176 100644 --- a/acceleration/internal/harness/tools.go +++ b/acceleration/internal/harness/tools.go @@ -5,6 +5,7 @@ import ( "errors" "fmt" "os" + "slices" "strings" "time" @@ -65,6 +66,9 @@ type ToolApproval struct { // Tools is the set a harness was configured with. type Tools struct { Tools []Tool `yaml:"tools"` + // Deferred are tools not offered until the model loads them with load_tool, after + // finding them with search_tool. The agent offers both whenever there are any. + Deferred []Tool `yaml:"-"` } // Lookup returns a tool by name. @@ -254,7 +258,7 @@ func (t Tools) SourceOrder() string { // Validate reports the first tool the harness could not use. func (t Tools) Validate() error { seen := map[string]struct{}{} - for _, tool := range t.Tools { + for _, tool := range slices.Concat(t.Tools, t.Deferred) { if tool.Name == "" { return stack.Wrap(errors.New("harness: every tool needs a name")) } diff --git a/acceleration/internal/llm/composition.go b/acceleration/internal/llm/composition.go index 62e52f8c19..22f1aa8ea2 100644 --- a/acceleration/internal/llm/composition.go +++ b/acceleration/internal/llm/composition.go @@ -43,6 +43,12 @@ func ToolTokens(tool Tool) int64 { return textTokens(tool.Name) + textTokens(tool.Description) + textTokens(string(schema)) } +// ToolUseTokens estimates what one call puts in front of the model: its name, the +// arguments it was given and the result it returned. +func ToolUseTokens(name, arguments, result string) int64 { + return textTokens(name) + textTokens(arguments) + textTokens(result) +} + // Compose estimates what a request's prompt is made of. func Compose(params ResponseParams) Composition { composed := Composition{Instructions: textTokens(params.Instructions)} diff --git a/acceleration/internal/node/workers.go b/acceleration/internal/node/workers.go index 6050361e05..9fbecbb37c 100644 --- a/acceleration/internal/node/workers.go +++ b/acceleration/internal/node/workers.go @@ -210,7 +210,7 @@ func (w *Workers) RunHosted(ctx context.Context, customerID, agentID string, cal } } if len(hosts) == 0 { - return nil, fmt.Errorf("dispatch: no worker is running %s now", call.Name) + return nil, dispatch.NoWorkerError{Tool: call.Name} } reply, err := w.ask(ctx, hosts[rand.IntN(len(hosts))], workRequest{ diff --git a/acceleration/internal/session/caller_tools.go b/acceleration/internal/session/caller_tools.go index 4285291bdc..c9ac40d333 100644 --- a/acceleration/internal/session/caller_tools.go +++ b/acceleration/internal/session/caller_tools.go @@ -5,6 +5,7 @@ import ( "errors" "fmt" "slices" + "strings" "github.com/GetStream/Vision-Agents/acceleration/internal/agent" persistent "github.com/GetStream/Vision-Agents/acceleration/internal/conversation" @@ -29,6 +30,9 @@ var ErrNoCallerTool = errors.New("session: the session's caller has no tool by t // in progress and every call in flight to finish (agent.SetTools), and the model is offered // them from the next turn. func (s *Session) AddTools(ctx context.Context, tools []harness.Tool) error { + if ungrouped := s.loading.ungrouped(tools); len(ungrouped) > 0 { + return stack.Wrap(fmt.Errorf("%w: %s", ErrToolNotConfigured, strings.Join(ungrouped, ", "))) + } return s.changeTools(ctx, func(declared []harness.Tool) ([]harness.Tool, error) { next := slices.Clone(declared) for _, tool := range tools { @@ -83,7 +87,7 @@ func (s *Session) changeTools(ctx context.Context, change func([]harness.Tool) ( return stack.Wrap(fmt.Errorf("%w: %s", ErrToolTaken, tool.Name)) } } - if err := s.current().SetTools(ctx, harness.Tools{Tools: s.offering(next)}, func() { + if err := s.current().SetTools(ctx, s.offering(next), func() { s.labelMu.Lock() s.spec.Tools = next if s.addedTools == nil { @@ -120,9 +124,9 @@ func (s *Session) callerTool(name string) (declared, added bool) { } // offering is everything the session offers with declared as the caller's tools, which go -// first as they did when it opened. -func (s *Session) offering(declared []harness.Tool) []harness.Tool { - return slices.Concat(declared, s.otherTools) +// first as they did when it opened, each loaded as its group or connector binding says. +func (s *Session) offering(declared []harness.Tool) harness.Tools { + return s.loading.arrange(slices.Concat(declared, s.otherTools)) } // toolPolicy is next with the caller's tools that asked to be cancelled by an interruption, diff --git a/acceleration/internal/session/connector_approval.go b/acceleration/internal/session/connector_approval.go index e79a931354..60b1684c1c 100644 --- a/acceleration/internal/session/connector_approval.go +++ b/acceleration/internal/session/connector_approval.go @@ -33,6 +33,9 @@ type approvals struct { await func(owner, callID, arguments string, approval persistent.ToolApproval) (string, string, bool) waits func(callID string) bool decide func(agent.ToolApprovalDecided) bool + // changed is told a call started waiting and how it was answered, for its state + // (Session.trackToolCall). Nil tells nobody. + changed func(agent.Event) // wait is how long a call waits, approvalWait when zero. wait time.Duration @@ -81,9 +84,9 @@ func (d *dispatcher) holdForApproval(ctx context.Context, r route, call llm.Tool // answer that comes at once finds it (answerApproval). a.mu.Lock() commandID, person, asked := "", "", false + title := fmt.Sprintf("Allow %s to run %s?", r.binding.Name, r.tool) if a.await != nil { - commandID, person, asked = a.await(d.spec.Caller.UserID, call.ID, call.Arguments, persistent.ToolApproval{ - Title: fmt.Sprintf("Allow %s to run %s?", r.binding.Name, r.tool)}) + commandID, person, asked = a.await(d.spec.Caller.UserID, call.ID, call.Arguments, persistent.ToolApproval{Title: title}) } if asked { if a.held == nil { @@ -92,6 +95,9 @@ func (d *dispatcher) holdForApproval(ctx context.Context, r route, call llm.Tool a.held[call.ID] = heldCall{commandID: commandID, asked: person, answer: answer} } a.mu.Unlock() + if asked { + a.tell(agent.ToolAwaitingApproval{ID: call.ID, TurnID: call.TurnID, Tool: call.Name, Question: title}) + } if !asked { return false, approvalText("unavailable", fmt.Sprintf("%s runs only once the user allows it, and nobody can be "+ "asked here, so it was not run. Tell the user it needs their approval in a chat of their own.", call.Name)), @@ -118,9 +124,11 @@ func (d *dispatcher) holdForApproval(ctx context.Context, r route, call llm.Tool return false, nil, failure{kind: store.InvocationClientTimeout}, stack.Wrap(fmt.Errorf("session: %s stopped waiting for its approval: %w", call.Name, ctx.Err())) case got := <-answer: - if !a.decide(agent.ToolApprovalDecided{ID: call.ID, TurnID: call.TurnID, Allowed: got.allowed, Summary: got.summary}) { + decided := agent.ToolApprovalDecided{ID: call.ID, TurnID: call.TurnID, Allowed: got.allowed, Summary: got.summary} + if !a.decide(decided) { return stopped() } + a.tell(decided) if got.allowed { return true, nil, failure{}, nil } @@ -128,7 +136,9 @@ func (d *dispatcher) holdForApproval(ctx context.Context, r route, call llm.Tool "run. Do not call it again unless they ask.", call.Name)), failure{kind: store.InvocationDenied, message: "declined by the person"}, nil case <-expiry.C: - a.decide(agent.ToolApprovalDecided{ID: call.ID, TurnID: call.TurnID, Summary: "Not allowed in time"}) + decided := agent.ToolApprovalDecided{ID: call.ID, TurnID: call.TurnID, Summary: "Not allowed in time"} + a.decide(decided) + a.tell(decided) return false, approvalText("declined", fmt.Sprintf("Nobody allowed %s in time, so it "+ "was declined and not run. Tell the user, and that they can ask again.", call.Name)), failure{kind: store.InvocationDenied, message: "declined by timeout"}, nil @@ -140,6 +150,13 @@ func (d *dispatcher) holdForApproval(ctx context.Context, r route, call llm.Tool } } +// tell passes event on to changed, when anything listens. +func (a *approvals) tell(event agent.Event) { + if a.changed != nil { + a.changed(event) + } +} + // answerApproval hands an answer to the call id held for a person, when it names the command // the call's reply answers and may says the one answering may answer for whom was asked. The // call is taken as it is answered, so it is answered once. diff --git a/acceleration/internal/session/hosted_tools.go b/acceleration/internal/session/hosted_tools.go index 2a870812c8..f246ad5977 100644 --- a/acceleration/internal/session/hosted_tools.go +++ b/acceleration/internal/session/hosted_tools.go @@ -2,6 +2,8 @@ package session import ( "context" + "errors" + "log/slog" "time" "github.com/GetStream/Vision-Agents/acceleration/internal/agent" @@ -27,8 +29,13 @@ func (m *Manager) HostTools(hosts ToolHosts) { m.hosts = hosts } // // The agent is the session's agent id, or failing that the name its config was opened by, // since a session opened by name carries an agent id of its own. The worker running a call -// reaches the session through conversation. -func (m *Manager) hostedTools(spec Spec, sessionID string, conversation dispatch.Progress, declared []harness.Tool, next agent.ToolRunner) ([]harness.Tool, agent.ToolRunner) { +// reaches the session through conversation. A tool no group of the session's config names +// is left out: the worker connected before the session opened, so there is nobody to refuse. +// +// Tools hosted for every agent (dispatch.EveryAgent) come after the agent's own, and only +// where a group of the session's config names them: registered once for the app, each agent +// takes the ones its agent.yaml enables, so the others are not worth a warning. +func (m *Manager) hostedTools(spec Spec, loading toolLoading, sessionID string, conversation dispatch.Progress, declared []harness.Tool, next agent.ToolRunner) ([]harness.Tool, agent.ToolRunner) { if m.hosts == nil { return declared, next } @@ -47,27 +54,50 @@ func (m *Manager) hostedTools(spec Spec, sessionID string, conversation dispatch for _, tool := range declared { taken[tool.Name] = true } - hosted := map[string]bool{} + // hosted is the agent id each offered tool is hosted under, which its calls go to. + hosted := map[string]string{} + offer := func(agentID string, tool dispatch.Tool) { + taken[tool.Name] = true + hosted[tool.Name] = agentID + offered := harness.Tool{ + Name: tool.Name, Description: tool.Description, Parameters: tool.Parameters, + Interruptible: tool.Interruptible, Timeout: tool.Timeout, + } + if tool.Approval != "" { + offered.Approval = &harness.ToolApproval{Title: tool.Approval} + } + declared = append(declared, offered) + } + var ungrouped []string for _, tool := range offered { if taken[tool.Name] { continue } - hosted[tool.Name] = true - offer := harness.Tool{ - Name: tool.Name, Description: tool.Description, Parameters: tool.Parameters, - Interruptible: tool.Interruptible, Timeout: tool.Timeout, + if !loading.grouped(tool.Name) { + ungrouped = append(ungrouped, tool.Name) + continue } - if tool.Approval != "" { - offer.Approval = &harness.ToolApproval{Title: tool.Approval} + offer(hostedAs, tool) + } + if loading.configured { + everyAgent, _ := m.hosts.HostedTools(spec.CustomerID, dispatch.EveryAgent) + for _, tool := range everyAgent { + if !taken[tool.Name] && loading.grouped(tool.Name) { + offer(dispatch.EveryAgent, tool) + } } - declared = append(declared, offer) + } + if len(ungrouped) > 0 { + m.logger.Warn("not offering tools a worker hosts that no tool group of the agent's config names", + "session", sessionID, "agent", hostedAs, "tools", ungrouped) } if len(hosted) == 0 { return declared, next } m.logger.Info("offering tools a worker hosts", "session", sessionID, "agent", hostedAs, "tools", len(hosted)) return declared, &hostedRunner{ - hosts: m.hosts, spec: spec, agentID: hostedAs, sessionID: sessionID, conversation: conversation, names: hosted, next: next, + hosts: m.hosts, spec: spec, sessionID: sessionID, conversation: conversation, hostedAs: hosted, next: next, + logger: m.logger, } } @@ -79,15 +109,17 @@ func (m *Manager) hostedTools(spec Spec, sessionID string, conversation dispatch type hostedRunner struct { hosts ToolHosts spec Spec - agentID string sessionID string conversation dispatch.Progress - names map[string]bool - next agent.ToolRunner + // hostedAs is the agent id each hosted tool's calls go to. + hostedAs map[string]string + next agent.ToolRunner + logger *slog.Logger } func (r *hostedRunner) Run(ctx context.Context, call llm.ToolCall) ([]llm.ContentPart, error) { - if !r.names[call.Name] { + agentID, ok := r.hostedAs[call.Name] + if !ok { return r.next.Run(ctx, call) } hosted := dispatch.ToolCall{ @@ -101,7 +133,12 @@ func (r *hostedRunner) Run(ctx context.Context, call llm.ToolCall) ([]llm.Conten } else if r.spec.Caller.UserID != "" { hosted.User = &dispatch.User{ID: r.spec.Caller.UserID} } - return r.hosts.RunHosted(ctx, r.spec.CustomerID, r.agentID, hosted) + parts, err := r.hosts.RunHosted(ctx, r.spec.CustomerID, agentID, hosted) + if gone := (dispatch.NoWorkerError{}); errors.As(err, &gone) { + r.logger.Warn("a tool a worker hosted when the session opened has no worker running it now; the agent carries on without it", + "session", r.sessionID, "agent", agentID, "tool", call.Name, "tool_call", call.ID) + } + return parts, err } // hostedProgress is a session as the worker running one of its hosted calls reaches it. The diff --git a/acceleration/internal/session/hosted_tools_test.go b/acceleration/internal/session/hosted_tools_test.go index bd4a40752a..25eb93f7af 100644 --- a/acceleration/internal/session/hosted_tools_test.go +++ b/acceleration/internal/session/hosted_tools_test.go @@ -9,15 +9,23 @@ import ( "github.com/GetStream/Vision-Agents/acceleration/internal/dispatch" "github.com/GetStream/Vision-Agents/acceleration/internal/harness" "github.com/GetStream/Vision-Agents/acceleration/internal/llm" + "github.com/GetStream/Vision-Agents/acceleration/internal/store" ) type fakeHosts struct { - ran []dispatch.ToolCall - agents []string + ran []dispatch.ToolCall + agents []string + everyAgent []dispatch.Tool } func (f *fakeHosts) HostedTools(customerID, agentID string) ([]dispatch.Tool, time.Duration) { - if customerID != "acme" || agentID != "stream-support" { + if customerID != "acme" { + return nil, 0 + } + if agentID == dispatch.EveryAgent { + return f.everyAgent, time.Minute + } + if agentID != "stream-support" { return nil, 0 } return []dispatch.Tool{ @@ -46,7 +54,8 @@ func TestASessionNamingAHostedAgentIsOfferedTheWorkersTools(t *testing.T) { declared := []harness.Tool{{Name: "search_docs", Description: "The browser's search"}} // The shape a browser opens: an agent id, no stored config, and its own search. - tools, runner := m.hostedTools(Spec{CustomerID: "acme", AgentID: "stream-support"}, "session-1", nil, declared, caller) + spec := Spec{CustomerID: "acme", AgentID: "stream-support"} + tools, runner := m.hostedTools(spec, newToolLoading(spec), "session-1", nil, declared, caller) if len(tools) != 2 || tools[0].Description != "The browser's search" || tools[1].Name != "investigate_sdk" { t.Fatalf("offered %+v", tools) @@ -70,8 +79,11 @@ func TestASessionOpenedByAgentNameIsOfferedTheWorkersTools(t *testing.T) { m := &Manager{logger: slog.Default(), hosts: hosts} // The shape a server SDK opens: a stored config found by name, and an agent id of its own. - spec := Spec{CustomerID: "acme", AgentID: "01JC-generated", ConfigID: "cfg-1", AgentName: "stream-support"} - tools, runner := m.hostedTools(spec, "session-1", nil, nil, &callerTools{}) + spec := Spec{ + CustomerID: "acme", AgentID: "01JC-generated", ConfigID: "cfg-1", AgentName: "stream-support", + ToolGroups: []store.ToolGroup{{Name: "support", Tools: []string{"investigate_sdk", "search_docs"}}}, + } + tools, runner := m.hostedTools(spec, newToolLoading(spec), "session-1", nil, nil, &callerTools{}) if len(tools) != 2 || tools[0].Name != "investigate_sdk" { t.Fatalf("offered %+v", tools) @@ -88,11 +100,13 @@ func TestASessionNamingAnotherAgentIsOfferedNothingHosted(t *testing.T) { m := &Manager{logger: slog.Default(), hosts: &fakeHosts{}} caller := &callerTools{} - tools, runner := m.hostedTools(Spec{CustomerID: "acme", AgentID: "sales"}, "s", nil, nil, caller) + sales := Spec{CustomerID: "acme", AgentID: "sales"} + tools, runner := m.hostedTools(sales, newToolLoading(sales), "s", nil, nil, caller) if len(tools) != 0 || runner != caller { t.Fatalf("offered %+v", tools) } - tools, _ = m.hostedTools(Spec{CustomerID: "acme", ConfigID: "stream-support"}, "s", nil, nil, caller) + byConfig := Spec{CustomerID: "acme", ConfigID: "stream-support"} + tools, _ = m.hostedTools(byConfig, newToolLoading(byConfig), "s", nil, nil, caller) if len(tools) != 0 { t.Fatalf("a session naming the agent only as a config was offered %+v", tools) } @@ -106,7 +120,8 @@ func TestAHostedCallCarriesTheConversationItWasMadeIn(t *testing.T) { m := &Manager{logger: slog.Default(), hosts: hosts} conversation := hostedConversation{} - _, runner := m.hostedTools(Spec{CustomerID: "acme", AgentID: "stream-support"}, "session-1", conversation, nil, nil) + spec := Spec{CustomerID: "acme", AgentID: "stream-support"} + _, runner := m.hostedTools(spec, newToolLoading(spec), "session-1", conversation, nil, nil) if _, err := runner.Run(context.Background(), llm.ToolCall{ID: "1", Name: "investigate_sdk"}); err != nil { t.Fatal(err) } @@ -115,3 +130,65 @@ func TestAHostedCallCarriesTheConversationItWasMadeIn(t *testing.T) { t.Errorf("the worker cannot reach the conversation: %+v", hosts.ran) } } + +func TestAWorkersToolNoGroupNamesIsNotOffered(t *testing.T) { + m := &Manager{logger: slog.Default(), hosts: &fakeHosts{}} + spec := Spec{ + CustomerID: "acme", AgentID: "01JC-generated", ConfigID: "cfg-1", AgentName: "stream-support", + ToolGroups: []store.ToolGroup{{Name: "docs", Tools: []string{"search_docs"}}}, + } + + tools, _ := m.hostedTools(spec, newToolLoading(spec), "session-1", nil, nil, &callerTools{}) + + if len(tools) != 1 || tools[0].Name != "search_docs" { + t.Fatalf("offered %+v", tools) + } +} + +func TestAToolHostedForEveryAgentIsOfferedWhereAToolGroupNamesIt(t *testing.T) { + hosts := &fakeHosts{everyAgent: []dispatch.Tool{ + {Name: "weather_lookup", Description: "The app's weather"}, + {Name: "refund_order", Description: "The app's refunds"}, + }} + m := &Manager{logger: slog.Default(), hosts: hosts} + + weather := Spec{ + CustomerID: "acme", AgentID: "01JC-generated", ConfigID: "cfg-1", AgentName: "concierge", + ToolGroups: []store.ToolGroup{{Name: "outside", Tools: []string{"weather_lookup"}}}, + } + tools, runner := m.hostedTools(weather, newToolLoading(weather), "session-1", nil, nil, &callerTools{}) + if len(tools) != 1 || tools[0].Name != "weather_lookup" { + t.Fatalf("offered %+v", tools) + } + if _, err := runner.Run(context.Background(), llm.ToolCall{ID: "1", Name: "weather_lookup"}); err != nil { + t.Fatal(err) + } + if len(hosts.agents) != 1 || hosts.agents[0] != dispatch.EveryAgent { + t.Errorf("the call went to the workers hosting %q, not those hosting for every agent", hosts.agents) + } + + unconfigured := Spec{CustomerID: "acme", AgentID: "browser-agent"} + if tools, _ := m.hostedTools(unconfigured, newToolLoading(unconfigured), "s", nil, nil, &callerTools{}); len(tools) != 0 { + t.Errorf("a session with no config to enable them was offered %+v", tools) + } +} + +func TestAnAgentsOwnHostedToolWinsOverOneHostedForEveryAgent(t *testing.T) { + hosts := &fakeHosts{everyAgent: []dispatch.Tool{{Name: "search_docs", Description: "The app's search"}}} + m := &Manager{logger: slog.Default(), hosts: hosts} + spec := Spec{ + CustomerID: "acme", AgentID: "01JC-generated", ConfigID: "cfg-1", AgentName: "stream-support", + ToolGroups: []store.ToolGroup{{Name: "docs", Tools: []string{"search_docs"}}}, + } + + tools, runner := m.hostedTools(spec, newToolLoading(spec), "session-1", nil, nil, &callerTools{}) + if len(tools) != 1 || tools[0].Description != "The worker's search" { + t.Fatalf("offered %+v", tools) + } + if _, err := runner.Run(context.Background(), llm.ToolCall{ID: "1", Name: "search_docs"}); err != nil { + t.Fatal(err) + } + if len(hosts.agents) != 1 || hosts.agents[0] != "stream-support" { + t.Errorf("the call went to the workers hosting %q", hosts.agents) + } +} diff --git a/acceleration/internal/session/logs.go b/acceleration/internal/session/logs.go index a11af9531a..bb128e9590 100644 --- a/acceleration/internal/session/logs.go +++ b/acceleration/internal/session/logs.go @@ -136,6 +136,28 @@ func (s *Session) recordLog(event Event) { row.Details["tool_call_id"] = e.ID row.Details["turn_id"] = e.TurnID row.Details["tool"] = e.Tool + case ToolCallChanged: + // Only a call's end: its start and its result have rows of their own. + if e.Phase != ToolCallFinished { + return + } + row.EventType = "tool_call_finished" + row.Source = "tool" + row.Message = e.State.Tool + " " + string(e.State.Status) + if e.State.Status == ToolCallFailed { + row.Severity = "error" + } + row.Details["tool_call_id"] = e.State.ID + row.Details["turn_id"] = e.State.TurnID + row.Details["tool"] = e.State.Tool + row.Details["status"] = string(e.State.Status) + row.Details["duration_ms"] = e.State.Duration.Milliseconds() + if !e.State.ApprovedAt.IsZero() { + row.Details["approved_at"] = e.State.ApprovedAt + } + if !e.State.RejectedAt.IsZero() { + row.Details["rejected_at"] = e.State.RejectedAt + } case agent.Delegated: row.EventType = "delegated" row.Message = "Delegated to " + e.Skill diff --git a/acceleration/internal/session/manager.go b/acceleration/internal/session/manager.go index 41da5a61b4..9ed3b9eebd 100644 --- a/acceleration/internal/session/manager.go +++ b/acceleration/internal/session/manager.go @@ -490,6 +490,15 @@ func (m *Manager) Create(ctx context.Context, spec Spec) (_ *Session, failure er } spec.ToolOutputRetention = floor.Shorter(spec.ToolOutputRetention) } + loading := newToolLoading(spec) + if ungrouped := loading.ungrouped(spec.Tools); len(ungrouped) > 0 { + // A conversation carried on from its row was opened with the tools its config named + // then. One the config no longer names is left out rather than ending the conversation. + if spec.Reopened.IsZero() { + return nil, stack.Wrap(fmt.Errorf("%w: %s", ErrToolNotConfigured, strings.Join(ungrouped, ", "))) + } + spec.Tools = slices.DeleteFunc(slices.Clone(spec.Tools), func(tool harness.Tool) bool { return !loading.grouped(tool.Name) }) + } // The Stream app the session acts in is settled once, before anything is done there, // and the whole session keeps it: its call, its transcript and the rows it writes. @@ -682,6 +691,7 @@ func (m *Manager) Create(ctx context.Context, spec Spec) (_ *Session, failure er } }() created := m.newSession(spec, conv, skills, opening, runtime) + created.loading = loading // A session held ends where it closes (Session.counted); one never held ends here, // unless it was turned away for a router shutting down or for a session already held // under its id (a redelivered hook): neither is a session that failed. @@ -697,6 +707,7 @@ func (m *Manager) Create(ctx context.Context, spec Spec) (_ *Session, failure er runtime.Cancel() created.Close() } + created.toolCalls.approves = created.approves created.tools = newBridge( time.Duration(spec.ToolTimeoutMs)*time.Millisecond, spec.Tools, @@ -710,7 +721,7 @@ func (m *Manager) Create(ctx context.Context, spec Spec) (_ *Session, failure er sources := map[string]ToolSource{} markTools(sources, tools, ToolFromSession) var callers agent.ToolRunner = created.tools - tools, callers = m.hostedTools(spec, created.id, hostedProgress{created}, tools, callers) + tools, callers = m.hostedTools(spec, loading, created.id, hostedProgress{created}, tools, callers) markTools(sources, tools[len(spec.Tools):], ToolFromWorker) tools, callers = moderationTools(spec, stream, tools, callers) if line != nil || m.reading(spec) || m.searching(spec) || spec.endsCalls() { @@ -720,6 +731,12 @@ func (m *Manager) Create(ctx context.Context, spec Spec) (_ *Session, failure er } tools = append(tools, spec.callTools(builtin.Tools)...) } + // A worker that is not connected, or an SDK that no longer defines a function, leaves + // the agent without it. The session still opens and does what it can. + if missing := loading.missing(tools); len(missing) > 0 { + m.logger.Warn("the agent's tool groups name tools nothing on this session runs; it carries on without them", + "session", created.id, "config", spec.ConfigID, "tools", missing) + } // A config's plugins and MCP servers are no longer opened, whatever its row still names: // connectors replaced them, and a save refuses them. @@ -756,6 +773,7 @@ func (m *Manager) Create(ctx context.Context, spec Spec) (_ *Session, failure er connectors.next = runner runner = connectors connectors.askIn(conv, created.chose) + connectors.approvals.changed = created.trackToolCall connectors.stepUps = newStepUps(m.options.Connectors.Consents, created.broadcast, m.logger) created.connectors = connectors created.closers = append(created.closers, connectors.Close, connectors.closeLogins) @@ -764,8 +782,8 @@ func (m *Manager) Create(ctx context.Context, spec Spec) (_ *Session, failure er created.toolSources = sources // Last, so every other tool's name is taken first and a child cannot shadow one. tools, runner = m.childAgents(spec, created, tools, runner) - if spec.ProgressiveTools { - tools, runner = progressively(tools, connectorTools, runner) + if loading.summarises() { + runner = newProgressiveRunner(runner, created.summarised) } created.otherTools = slices.Clone(tools[len(spec.Tools):]) @@ -863,7 +881,7 @@ func (m *Manager) Create(ctx context.Context, spec Spec) (_ *Session, failure er Ender: m.ender(created, spec, stream), EndCallWhen: spec.endCallWhen(), ToolRunner: runner, - Tools: harness.Tools{Tools: created.offering(spec.Tools)}, + Tools: created.offering(spec.Tools), Sandbox: box, Publish: publisher(conv), Tasks: spec.Tasks, diff --git a/acceleration/internal/session/progressive_tools.go b/acceleration/internal/session/progressive_tools.go index e38a4df1d8..e5434240a2 100644 --- a/acceleration/internal/session/progressive_tools.go +++ b/acceleration/internal/session/progressive_tools.go @@ -4,7 +4,6 @@ import ( "context" "encoding/json" "fmt" - "slices" "strings" "sync" @@ -13,48 +12,30 @@ import ( "github.com/GetStream/Vision-Agents/acceleration/internal/llm" ) -// summaryLimit caps the description a deferred tool is offered with, in runes. -const summaryLimit = 200 - -// firstCallNote ends a deferred tool's description, so the model expects its first call to +// firstCallNote ends a summarised tool's description, so the model expects its first call to // come back with instructions rather than a result. const firstCallNote = "Its first call runs nothing and returns how to use it." -// progressively offers the deferred tools among tools by the first line of what each does -// and its arguments without their descriptions, and puts a runner in front of next that -// answers the first call to each with all of it instead of running it. A server that writes -// a page of examples into a tool's description then costs that page only in the -// conversations that use the tool, at the price of one more model turn when they do. -func progressively(tools, deferred []harness.Tool, next agent.ToolRunner) ([]harness.Tool, agent.ToolRunner) { - if len(deferred) == 0 { - return tools, next - } - runner := &progressiveRunner{next: next, full: map[string]harness.Tool{}, described: map[string]bool{}} - for _, tool := range deferred { - runner.full[tool.Name] = tool - } - offered := slices.Clone(tools) - for i, tool := range offered { - if _, ok := runner.full[tool.Name]; ok { - offered[i] = briefTool(tool) - } - } - return offered, runner -} - -// progressiveRunner answers the first call to a deferred tool with how to use it, and -// hands every other call to next. +// progressiveRunner answers the first call to a summarised tool (store.ToolLoadingSummary) +// with how to use it, and hands every other call to next. A server that writes a page of +// examples into a tool's description then costs that page only in the conversations that use +// the tool, at the price of one more model turn when they do. type progressiveRunner struct { next agent.ToolRunner - // full are the deferred tools as their servers describe them, by name. - full map[string]harness.Tool + // full is the summarised tool called name as its server or SDK describes it, read when + // it is called since a caller's tools change while the session runs. + full func(name string) (harness.Tool, bool) mu sync.Mutex described map[string]bool } +func newProgressiveRunner(next agent.ToolRunner, full func(string) (harness.Tool, bool)) *progressiveRunner { + return &progressiveRunner{next: next, full: full, described: map[string]bool{}} +} + func (r *progressiveRunner) Run(ctx context.Context, call llm.ToolCall) ([]llm.ContentPart, error) { - if tool, ok := r.full[call.Name]; ok && r.firstCall(call.Name) { + if tool, ok := r.full(call.Name); ok && r.firstCall(call.Name) { return llm.TextParts(howToUse(tool)), nil } return r.next.Run(ctx, call) @@ -76,34 +57,15 @@ func howToUse(tool harness.Tool) string { tool.Name, tool.Description, schema) } -// briefTool is tool as a deferred one is offered. +// briefTool is tool as a summarised one is offered. func briefTool(tool harness.Tool) harness.Tool { - tool.Description = strings.TrimSpace(summary(tool.Description) + " " + firstCallNote) + tool.Description = strings.TrimSpace(harness.Summary(tool.Description) + " " + firstCallNote) if tool.Parameters != nil { tool.Parameters = bareSchema(tool.Parameters) } return tool } -// summary is the first line of a description, cut at a word to summaryLimit runes. -func summary(description string) string { - var first string - for line := range strings.Lines(description) { - if first = strings.TrimSpace(line); first != "" { - break - } - } - runes := []rune(first) - if len(runes) <= summaryLimit { - return first - } - cut := string(runes[:summaryLimit]) - if space := strings.LastIndex(cut, " "); space > 0 { - cut = cut[:space] - } - return cut + "…" -} - // bareSchema is a JSON Schema without the words written for a reader: every description, // title and example, at every depth. What a call has to look like stays: the property // names, types, enums and what is required. diff --git a/acceleration/internal/session/progressive_tools_test.go b/acceleration/internal/session/progressive_tools_test.go index ae4bc0f85f..5c1d82dc0e 100644 --- a/acceleration/internal/session/progressive_tools_test.go +++ b/acceleration/internal/session/progressive_tools_test.go @@ -2,6 +2,7 @@ package session import ( "context" + "slices" "strings" "testing" @@ -10,19 +11,24 @@ import ( "github.com/GetStream/Vision-Agents/acceleration/internal/agent" "github.com/GetStream/Vision-Agents/acceleration/internal/harness" "github.com/GetStream/Vision-Agents/acceleration/internal/llm" + "github.com/GetStream/Vision-Agents/acceleration/internal/options" + "github.com/GetStream/Vision-Agents/acceleration/internal/store" ) -// ProgressiveToolsSuite covers offering tools by a summary and handing their full -// description over on the first call. -type ProgressiveToolsSuite struct { +// ToolLoadingSuite covers how a session under an agent config offers its tools: only the +// functions a tool group names, each whole, by a summary whose full description the first +// call hands over, or held back for load_tool. +type ToolLoadingSuite struct { suite.Suite - ran *ranTools - update harness.Tool - lookup harness.Tool + ran *ranTools + update harness.Tool + lookup harness.Tool + refund harness.Tool + loading toolLoading } -func TestProgressiveToolsSuite(t *testing.T) { - suite.Run(t, new(ProgressiveToolsSuite)) +func TestToolLoadingSuite(t *testing.T) { + suite.Run(t, new(ToolLoadingSuite)) } type ranTools struct{ names []string } @@ -32,7 +38,7 @@ func (r *ranTools) Run(_ context.Context, call llm.ToolCall) ([]llm.ContentPart, return llm.TextParts("ran " + call.Name), nil } -func (s *ProgressiveToolsSuite) SetupTest() { +func (s *ToolLoadingSuite) SetupTest() { s.ran = &ranTools{} s.update = harness.Tool{ Name: "sentry__update_issue", @@ -55,23 +61,40 @@ func (s *ProgressiveToolsSuite) SetupTest() { }, } s.lookup = harness.Tool{Name: "lookup", Description: "Look it up.\nIn the knowledge base."} + s.refund = harness.Tool{Name: "refund_booking", Description: "Refund a booking."} + s.loading = newToolLoading(Spec{ + ConfigID: "cfg-1", + ToolGroups: []store.ToolGroup{ + {Name: "knowledge", Tools: []string{"lookup"}, OutputRetention: "1h"}, + {Name: "refunds", Loading: store.ToolLoadingDeferred, Tools: []string{"refund_booking"}}, + }, + ConnectorBindings: []store.ConnectorBinding{{Name: "sentry", Loading: store.ToolLoadingSummary}}, + }) } -func (s *ProgressiveToolsSuite) offer() ([]harness.Tool, agent.ToolRunner) { - return progressively([]harness.Tool{s.lookup, s.update}, []harness.Tool{s.update}, s.ran) +func (s *ToolLoadingSuite) offer() (harness.Tools, agent.ToolRunner) { + tools := []harness.Tool{s.lookup, s.update, s.refund} + full := func(name string) (harness.Tool, bool) { + at := slices.IndexFunc(tools, named(name)) + if at < 0 || s.loading.mode(name) != store.ToolLoadingSummary { + return harness.Tool{}, false + } + return tools[at], true + } + return s.loading.arrange(tools), newProgressiveRunner(s.ran, full) } -func (s *ProgressiveToolsSuite) call(runner agent.ToolRunner, name string) string { +func (s *ToolLoadingSuite) call(runner agent.ToolRunner, name string) string { parts, err := runner.Run(context.Background(), llm.ToolCall{ID: "call", Name: name, Arguments: `{"status":"resolved"}`}) s.Require().NoError(err) s.Require().Len(parts, 1) return parts[0].Text } -func (s *ProgressiveToolsSuite) TestADeferredToolIsOfferedByItsFirstLineAndTheShapeOfItsArguments() { +func (s *ToolLoadingSuite) TestASummarisedToolIsOfferedByItsFirstLineAndTheShapeOfItsArguments() { tools, _ := s.offer() - s.Equal("Update an issue's status or assignment in Sentry. "+firstCallNote, tools[1].Description) + s.Equal("Update an issue's status or assignment in Sentry. "+firstCallNote, tools.Tools[1].Description) s.Equal(map[string]any{ "type": "object", "properties": map[string]any{ @@ -80,10 +103,10 @@ func (s *ProgressiveToolsSuite) TestADeferredToolIsOfferedByItsFirstLineAndTheSh "assignees": map[string]any{"type": "array", "items": map[string]any{"type": "string"}}, }, "required": []any{"status"}, - }, tools[1].Parameters) + }, tools.Tools[1].Parameters) } -func (s *ProgressiveToolsSuite) TestTheFirstCallReturnsHowToUseTheToolAndRunsNothing() { +func (s *ToolLoadingSuite) TestTheFirstCallReturnsHowToUseTheToolAndRunsNothing() { _, runner := s.offer() answer := s.call(runner, "sentry__update_issue") @@ -94,7 +117,7 @@ func (s *ProgressiveToolsSuite) TestTheFirstCallReturnsHowToUseTheToolAndRunsNot s.Contains(answer, `"description":"The new status of the issue."`) } -func (s *ProgressiveToolsSuite) TestTheCallAfterThatRuns() { +func (s *ToolLoadingSuite) TestTheCallAfterThatRuns() { _, runner := s.offer() s.call(runner, "sentry__update_issue") @@ -104,28 +127,56 @@ func (s *ProgressiveToolsSuite) TestTheCallAfterThatRuns() { s.Equal([]string{"sentry__update_issue"}, s.ran.names) } -func (s *ProgressiveToolsSuite) TestAToolThatIsNotDeferredIsOfferedWholeAndRunsAtOnce() { +func (s *ToolLoadingSuite) TestAFullToolIsOfferedWholeAndRunsAtOnce() { tools, runner := s.offer() answer := s.call(runner, "lookup") - s.Equal(s.lookup, tools[0]) + s.Equal(s.lookup, tools.Tools[0]) s.Equal("ran lookup", answer) } -func (s *ProgressiveToolsSuite) TestNothingDeferredLeavesTheToolsAndTheRunnerAsTheyWere() { - tools, runner := progressively([]harness.Tool{s.update}, nil, s.ran) +func (s *ToolLoadingSuite) TestADeferredToolIsHeldBackWhole() { + tools, _ := s.offer() - s.Equal([]harness.Tool{s.update}, tools) - s.Same(s.ran, runner) + s.Equal([]harness.Tool{s.refund}, tools.Deferred) + s.False(slices.ContainsFunc(tools.Tools, named("refund_booking")), "offered before it was loaded") } -func (s *ProgressiveToolsSuite) TestALongFirstLineIsCutAtAWord() { +func (s *ToolLoadingSuite) TestALongFirstLineIsCutAtAWord() { s.update.Description = strings.Repeat("word ", 60) tools, _ := s.offer() - summary, _ := strings.CutSuffix(tools[1].Description, " "+firstCallNote) + summary, _ := strings.CutSuffix(tools.Tools[1].Description, " "+firstCallNote) s.True(strings.HasSuffix(summary, "word…"), summary) - s.LessOrEqual(len([]rune(summary)), summaryLimit+1) + s.Less(len([]rune(summary)), len([]rune(strings.Repeat("word ", 60)))) +} + +func (s *ToolLoadingSuite) TestAFunctionNoGroupNamesIsUngroupedUnderAConfig() { + s.Equal([]string{"cancel_booking"}, + s.loading.ungrouped([]harness.Tool{s.lookup, {Name: "cancel_booking"}, {Name: agent.VideoFramesTool}})) +} + +func (s *ToolLoadingSuite) TestAGroupedFunctionNothingOnTheSessionRunsIsMissing() { + s.Equal([]string{"refund_booking"}, s.loading.missing([]harness.Tool{s.lookup, s.update})) + s.Empty(s.loading.missing([]harness.Tool{s.refund, s.lookup})) +} + +func (s *ToolLoadingSuite) TestASessionWithoutAConfigOffersWhatItsCallerDeclares() { + loading := newToolLoading(Spec{CustomerID: "acme", AgentID: "browser"}) + + s.Empty(loading.ungrouped([]harness.Tool{{Name: "cancel_booking"}})) + s.Equal(store.ToolLoadingFull, loading.mode("cancel_booking")) +} + +func (s *ToolLoadingSuite) TestAToolKeepsWhatItGaveBackAsLongAsItsGroupSays() { + s.Equal(options.Retention("1h"), s.loading.retention("lookup")) + s.Equal(options.Retention(""), s.loading.retention("refund_booking")) +} + +func (s *ToolLoadingSuite) TestAConnectorsToolIsOfferedAsItsBindingSays() { + s.Equal(store.ToolLoadingSummary, s.loading.mode("sentry__list_issues")) + s.Equal(store.ToolLoadingFull, s.loading.mode("linear__list_issues"), "a connector the config does not bind") + s.Equal(store.ToolLoadingFull, s.loading.mode("sentry"), "a function named like an alias") } diff --git a/acceleration/internal/session/records_test.go b/acceleration/internal/session/records_test.go index 1d30563003..59401c7e0e 100644 --- a/acceleration/internal/session/records_test.go +++ b/acceleration/internal/session/records_test.go @@ -14,6 +14,7 @@ import ( "github.com/GetStream/Vision-Agents/acceleration/internal/agent" "github.com/GetStream/Vision-Agents/acceleration/internal/auth" + "github.com/GetStream/Vision-Agents/acceleration/internal/harness" "github.com/GetStream/Vision-Agents/acceleration/internal/llm" "github.com/GetStream/Vision-Agents/acceleration/internal/options" "github.com/GetStream/Vision-Agents/acceleration/internal/policy" @@ -181,6 +182,37 @@ func (s *RecordSuite) TestItemsKeepTheOrderTheyHappenedIn() { } } +func (s *RecordSuite) TestACallRecordsWhatItsToolsDefinitionAndItsUseCost() { + spec := Spec{ToolGroups: []store.ToolGroup{{Name: "orders", Loading: store.ToolLoadingSummary, Tools: []string{"lookup_order"}}}} + s.session.spec = spec + s.session.loading = newToolLoading(spec) + s.session.otherTools = []harness.Tool{{ + Name: "lookup_order", + Description: "Finds an order.\n\nPass the order number exactly as the customer said it, " + + "with its letters, and never guess one.", + Parameters: map[string]any{"type": "object", "properties": map[string]any{ + "order": map[string]any{"type": "string", "description": "The order number, such as A-1042."}, + }}, + }} + s.took( + agent.Responding{TurnID: "turn-1", Prompt: "Where is A-1042?"}, + agent.ToolStarted{TurnID: "turn-1", ID: "call-1", Tool: "lookup_order"}, + agent.ToolRan{TurnID: "turn-1", ID: "call-1", Tool: "lookup_order", + Arguments: `{"order":"A-1042"}`, Result: "shipped on Monday"}, + agent.ToolStarted{TurnID: "turn-1", ID: "call-2", Tool: "end_call"}, + agent.Responded{TurnID: "turn-1", Text: "It shipped on Monday."}, + ) + + call, result, builtIn := s.held.items[1].Payload, s.held.items[2].Payload, s.held.items[3].Payload + s.Equal(store.ToolLoadingSummary, call["loading"]) + whole, summary := call["definition_tokens"].(int64), call["summary_tokens"].(int64) + s.Positive(summary) + s.Greater(whole, summary, "the summary leaves out the description's detail and the schema's words") + // lookup_order, {"order":"A-1042"} and shipped on Monday are 12, 18 and 17 bytes: 3, 5 and 5 tokens. + s.EqualValues(13, result["tokens"]) + s.NotContains(builtIn, "definition_tokens", "a tool the session does not describe has nothing to weigh") +} + // The tool_ran frame of a failed tool is redacted (sessionws errorText); the tool_result item // the client lists later says the same, not the key the tool's provider echoed. func (s *RecordSuite) TestAToolsErrorIsRecordedWithoutTheKeyItEchoed() { @@ -605,7 +637,8 @@ func (s *RecordSuite) TestAToolOutputRetentionOfNoneRecordsNoneOfIt() { s.Equal(store.ItemToolResult, result.Kind) s.Equal("orders", result.ToolName) s.Empty(result.Text) - s.Equal(map[string]any{"call_id": "call-1", "error": "redacted", "redacted": true}, result.Payload) + // What the call cost in tokens is a count tool usage reads, not what the tool was given. + s.Equal(map[string]any{"call_id": "call-1", "error": "redacted", "redacted": true, "tokens": int64(14)}, result.Payload) s.Nil(result.RedactAt, "nothing is left to redact later") s.Equal("It ships tomorrow.", s.held.items[4].Text) } diff --git a/acceleration/internal/session/session.go b/acceleration/internal/session/session.go index 1fc8f3fcf5..846df38bd2 100644 --- a/acceleration/internal/session/session.go +++ b/acceleration/internal/session/session.go @@ -190,6 +190,7 @@ type Session struct { typed []string typedTurns map[string]struct{} tools *bridge + toolCalls toolCalls // calls records the call's row when it moves onto other models. Nil when the // deployment keeps no calls. calls *callRecorder @@ -287,8 +288,12 @@ type Session struct { toolSources map[string]ToolSource // otherTools are what the session offers besides its caller's own tools, which go first: // the workers', the router's, the connectors' and other agents'. Set before the session - // is served and only read after; the caller's change (AddTools, RemoveTool). + // is served and only read after; the caller's change (AddTools, RemoveTool). Each is + // held whole, and offered as loading says. otherTools []harness.Tool + // loading is how each tool is offered and how long what it was given and gave back is + // kept. Set before the session is served and only read after. + loading toolLoading // connectors is the session's connector dispatcher, nil when its config binds none. A // consent finished for one of its logins is handed back to it (ConnectorConsentFinished). connectors *dispatcher @@ -1842,7 +1847,9 @@ func (s *Session) DecideCommandTool(id, commandID, turnID string, allowed bool, return false } s.tools.decide(id, turnID) - s.persisted.Observe(agent.ToolApprovalDecided{ID: id, TurnID: turnID, Allowed: allowed, Summary: summary}) + decided := agent.ToolApprovalDecided{ID: id, TurnID: turnID, Allowed: allowed, Summary: summary} + s.persisted.Observe(decided) + s.trackToolCall(decided) return true } @@ -1957,6 +1964,7 @@ func (s *Session) consume(held *agent.Agent, transcript Transcript, run *presenc s.remember(event) s.record(event) s.broadcast(event) + s.trackToolCall(event) s.name(event) } s.heard(held) @@ -2027,11 +2035,14 @@ func (s *Session) write(event Event) { s.openTurn(typed.TurnID, typed.Prompt) } case agent.ToolStarted: - s.item(typed.TurnID, store.ItemToolCall, "", typed.Tool, map[string]any{ - "call_id": typed.ID, "product": typed.Product, "sdk": typed.SDK, - }) + payload := map[string]any{"call_id": typed.ID, "product": typed.Product, "sdk": typed.SDK} + s.describeOffer(payload, typed.Tool) + s.item(typed.TurnID, store.ItemToolCall, "", typed.Tool, payload) case agent.ToolRan: - payload := map[string]any{"call_id": typed.ID, "arguments": typed.Arguments} + payload := map[string]any{ + "call_id": typed.ID, "arguments": typed.Arguments, + "tokens": llm.ToolUseTokens(typed.Tool, typed.Arguments, typed.Result), + } if typed.Err != nil { payload["error"] = store.SafeLogText(typed.Err.Error()) } @@ -2219,7 +2230,7 @@ func (s *Session) item(turnID, kind, text, tool string, payload map[string]any) Kind: kind, Text: text, ToolName: tool, Payload: payload, At: time.Now().UTC(), } if kind == store.ItemToolCall || kind == store.ItemToolResult { - switch window, kept := s.spec.ToolOutputRetention.Window(); { + switch window, kept := s.spec.ToolOutputRetention.Shorter(s.loading.retention(tool)).Window(); { case kept && window == 0: item.Redact() case kept: diff --git a/acceleration/internal/session/spec.go b/acceleration/internal/session/spec.go index 6107373c6a..29f9bb6f7a 100644 --- a/acceleration/internal/session/spec.go +++ b/acceleration/internal/session/spec.go @@ -286,13 +286,14 @@ type Spec struct { // session does what it did before the cards existed. EpisodeCards bool EpisodeCardsRecord bool - // ProgressiveTools offers plugin, MCP server and connector tools by a summary, and - // answers the first call to each with its full description instead of running it. - ProgressiveTools bool - // ToolOutputRetention is how long the turns this session records keep what its tools - // were given and gave back: options.RetentionNone for not at all, a duration, or empty - // for as long as the session. From the agent's config, and the manager shortens it to - // the app's and the organization's policies when the session is created. + // ToolGroups are the agent config's groups of the functions an SDK provides: a session + // under a config offers only the functions they name, each loaded as its group says. + ToolGroups []store.ToolGroup + // ToolOutputRetention is how long the turns this session records keep what any of its + // tools were given and gave back: options.RetentionNone for not at all, a duration, or + // empty for as long as the session. The manager sets it to the app's and the + // organization's policies when the session is created; a tool's group or connector + // binding can shorten it for that tool (toolLoading.retention). ToolOutputRetention options.Retention // ResumeHistory is how a conversation the session carries on is handed back to the // model: the session request's, else the agent config's. Zero is the router's default. @@ -453,7 +454,7 @@ func FromConfig(config store.AgentConfig) Spec { DispatchText: config.DispatchText, EpisodeCards: config.EpisodeCards, EpisodeCardsRecord: config.EpisodeCardsRecord, - ProgressiveTools: config.ProgressiveTools, + ToolGroups: config.ToolGroups, ResumeHistory: config.ResumeHistory, FloorControl: config.FloorControl, Scheduling: config.Scheduling, @@ -461,8 +462,6 @@ func FromConfig(config store.AgentConfig) Spec { Tags: routing.Tags(config.Tags), Team: configTeam(config.Tags), ConfigTeam: strings.TrimSpace(config.Tags[TeamTag]), - // The config's alone until Manager.Create holds it to the policies. - ToolOutputRetention: config.ToolOutputRetention, } } diff --git a/acceleration/internal/session/tool_call_state.go b/acceleration/internal/session/tool_call_state.go new file mode 100644 index 0000000000..61264eaf38 --- /dev/null +++ b/acceleration/internal/session/tool_call_state.go @@ -0,0 +1,196 @@ +package session + +import ( + "context" + "encoding/json" + "errors" + "sync" + "time" + + "github.com/GetStream/Vision-Agents/acceleration/internal/agent" +) + +// ToolCallStatus is where one tool call stands. +type ToolCallStatus string + +const ( + ToolCallInProgress ToolCallStatus = "in_progress" + // ToolCallWaiting is a call held for a person's approval. + ToolCallWaiting ToolCallStatus = "waiting" + ToolCallSuccess ToolCallStatus = "success" + ToolCallFailed ToolCallStatus = "failed" + ToolCallCancelled ToolCallStatus = "cancelled" +) + +// ToolCallPhase says which of a call's changes a ToolCallChanged is: its first, one while it +// runs, or its last. +type ToolCallPhase string + +const ( + ToolCallStarted ToolCallPhase = "started" + ToolCallProgress ToolCallPhase = "progress" + ToolCallFinished ToolCallPhase = "finished" +) + +// ToolCallState is everything a client shows of one tool call, as it is now. +type ToolCallState struct { + ID string + Tool string + TurnID string + Status ToolCallStatus + // Progress is the last thing said about the call while it runs, such as "Searching for + // flights", or the question it waits on. + Progress string + StartedAt time.Time + // Duration is from StartedAt to this change, or to the end for a finished call. + Duration time.Duration + ApprovedAt time.Time + RejectedAt time.Time + // Error is why a call failed, or what a declined one shows, such as "Not allowed in time". + Error string +} + +// ToolCallChanged is one change to a tool call's state, sent to every watcher. +type ToolCallChanged struct { + Phase ToolCallPhase + State ToolCallState +} + +// toolCalls follows the session's tool calls from their agent's events to the state a client +// shows. A call is forgotten once it finishes, so a late event about it changes nothing. +type toolCalls struct { + // approves reports whether a call to a tool waits for a person before it runs. + approves func(tool string) bool + + mu sync.Mutex + calls map[string]*ToolCallState + // early are the calls a connector began holding for approval before their agent's + // ToolStarted came through, which the two report on separate goroutines. + early map[string]agent.ToolAwaitingApproval +} + +// observe applies event and returns the change it made, reporting false for none. +func (t *toolCalls) observe(event agent.Event) (ToolCallChanged, bool) { + t.mu.Lock() + defer t.mu.Unlock() + now := time.Now().UTC() + switch typed := event.(type) { + case agent.ToolStarted: + if typed.Tool == agent.ConfirmTool || t.calls[typed.ID] != nil { + return ToolCallChanged{}, false + } + state := &ToolCallState{ID: typed.ID, Tool: typed.Tool, TurnID: typed.TurnID, Status: ToolCallInProgress, + Progress: typed.PreSpeech, StartedAt: typed.StartedAt} + if state.StartedAt.IsZero() { + state.StartedAt = now + } + if t.approves != nil && t.approves(typed.Tool) { + state.Status = ToolCallWaiting + } + if waiting, ok := t.early[typed.ID]; ok { + delete(t.early, typed.ID) + state.Status, state.Progress = ToolCallWaiting, waiting.Question + } + if t.calls == nil { + t.calls = map[string]*ToolCallState{} + } + t.calls[typed.ID] = state + return ToolCallChanged{Phase: ToolCallStarted, State: *state}, true + case agent.ToolAwaitingApproval: + if t.calls[typed.ID] == nil { + if t.early == nil { + t.early = map[string]agent.ToolAwaitingApproval{} + } + t.early[typed.ID] = typed + return ToolCallChanged{}, false + } + return t.changeLocked(typed.ID, now, func(state *ToolCallState) { + state.Status = ToolCallWaiting + if typed.Question != "" { + state.Progress = typed.Question + } + }) + case agent.ToolUpdated: + return t.changeLocked(typed.ID, now, func(state *ToolCallState) { state.Progress = typed.Text }) + case agent.ToolApprovalDecided: + if !typed.Allowed { + return t.finishLocked(typed.ID, now, ToolCallCancelled, typed.Summary) + } + return t.changeLocked(typed.ID, now, func(state *ToolCallState) { + state.Status, state.ApprovedAt = ToolCallInProgress, now + }) + case agent.ToolRan: + if typed.Asked { + return ToolCallChanged{}, false + } + id := typed.ID + // An approved call runs inside the confirmation that answered its question. + if typed.Tool == agent.ConfirmTool { + var answer struct { + ToolCallID string `json:"tool_call_id"` + } + if json.Unmarshal([]byte(typed.Arguments), &answer) != nil { + return ToolCallChanged{}, false + } + id = answer.ToolCallID + } + switch { + case errors.Is(typed.Err, context.Canceled): + return t.finishLocked(id, now, ToolCallCancelled, "") + case typed.Err != nil: + return t.finishLocked(id, now, ToolCallFailed, typed.Err.Error()) + } + if state := t.calls[id]; state != nil && state.Status == ToolCallWaiting { + return t.finishLocked(id, now, ToolCallCancelled, "") + } + return t.finishLocked(id, now, ToolCallSuccess, "") + } + return ToolCallChanged{}, false +} + +// changeLocked applies change to a call still running. The caller holds t.mu. +func (t *toolCalls) changeLocked(id string, now time.Time, change func(*ToolCallState)) (ToolCallChanged, bool) { + state := t.calls[id] + if state == nil { + return ToolCallChanged{}, false + } + before := *state + change(state) + if *state == before { + return ToolCallChanged{}, false + } + state.Duration = now.Sub(state.StartedAt) + return ToolCallChanged{Phase: ToolCallProgress, State: *state}, true +} + +// finishLocked ends a call as status and forgets it. A call declined while it waited is +// rejected now. The caller holds t.mu. +func (t *toolCalls) finishLocked(id string, now time.Time, status ToolCallStatus, reason string) (ToolCallChanged, bool) { + state := t.calls[id] + if state == nil { + return ToolCallChanged{}, false + } + delete(t.calls, id) + if status == ToolCallCancelled && state.Status == ToolCallWaiting { + state.RejectedAt = now + } + state.Status, state.Error, state.Duration = status, reason, now.Sub(state.StartedAt) + return ToolCallChanged{Phase: ToolCallFinished, State: *state}, true +} + +// trackToolCall sends every watcher the change event made to a tool call, if it made one. +func (s *Session) trackToolCall(event agent.Event) { + if changed, ok := s.toolCalls.observe(event); ok { + s.broadcast(changed) + } +} + +// approves reports whether a call to the caller's tool waits for a person before it runs. +func (s *Session) approves(tool string) bool { + for _, declared := range s.declaredTools() { + if declared.Name == tool { + return declared.Approval != nil + } + } + return false +} diff --git a/acceleration/internal/session/tool_call_state_test.go b/acceleration/internal/session/tool_call_state_test.go new file mode 100644 index 0000000000..1ccb5081e1 --- /dev/null +++ b/acceleration/internal/session/tool_call_state_test.go @@ -0,0 +1,151 @@ +package session + +import ( + "context" + "errors" + "fmt" + "testing" + "time" + + "github.com/stretchr/testify/suite" + + "github.com/GetStream/Vision-Agents/acceleration/internal/agent" +) + +type ToolCallStateSuite struct { + suite.Suite + calls *toolCalls + began time.Time +} + +func TestToolCallState(t *testing.T) { + suite.Run(t, new(ToolCallStateSuite)) +} + +func (s *ToolCallStateSuite) SetupTest() { + s.calls = &toolCalls{approves: func(tool string) bool { return tool == "book_flight" }} + s.began = time.Now().UTC().Add(-time.Second) +} + +// apply feeds the tracker event and returns the change it made. +func (s *ToolCallStateSuite) apply(event agent.Event) (ToolCallChanged, bool) { + return s.calls.observe(event) +} + +func (s *ToolCallStateSuite) start(id, tool string) ToolCallChanged { + changed, ok := s.apply(agent.ToolStarted{ID: id, TurnID: "turn-1", Tool: tool, StartedAt: s.began}) + s.Require().True(ok) + return changed +} + +func (s *ToolCallStateSuite) TestACallThatNeedsApprovalWaitsThenRunsOnceAllowedAndSucceeds() { + started := s.start("call-1", "book_flight") + s.Equal(ToolCallStarted, started.Phase) + s.Equal(ToolCallWaiting, started.State.Status) + + allowed, ok := s.apply(agent.ToolApprovalDecided{ID: "call-1", TurnID: "turn-1", Allowed: true}) + s.Require().True(ok) + s.Equal(ToolCallProgress, allowed.Phase) + s.Equal(ToolCallInProgress, allowed.State.Status) + s.False(allowed.State.ApprovedAt.IsZero()) + + finished, ok := s.apply(agent.ToolRan{ID: "call-1", TurnID: "turn-1", Tool: "book_flight", Result: "booked"}) + s.Require().True(ok) + s.Equal(ToolCallFinished, finished.Phase) + s.Equal(ToolCallSuccess, finished.State.Status) + s.Equal(allowed.State.ApprovedAt, finished.State.ApprovedAt) + s.GreaterOrEqual(finished.State.Duration, time.Second) +} + +func (s *ToolCallStateSuite) TestADeclinedCallFinishesCancelledWithWhenItWasRejected() { + s.start("call-1", "book_flight") + + declined, ok := s.apply(agent.ToolApprovalDecided{ID: "call-1", TurnID: "turn-1", Summary: "Not booked"}) + + s.Require().True(ok) + s.Equal(ToolCallFinished, declined.Phase) + s.Equal(ToolCallCancelled, declined.State.Status) + s.False(declined.State.RejectedAt.IsZero()) + s.True(declined.State.ApprovedAt.IsZero()) + s.Equal("Not booked", declined.State.Error) + _, late := s.apply(agent.ToolRan{ID: "call-1", Tool: "book_flight", Err: errors.New("declined")}) + s.False(late, "a call is finished once") +} + +func (s *ToolCallStateSuite) TestAnUpdateWhileTheCallRunsIsItsProgress() { + s.start("call-1", "search_flights") + + progress, ok := s.apply(agent.ToolUpdated{ID: "call-1", Tool: "search_flights", Text: "Searching for flights"}) + + s.Require().True(ok) + s.Equal(ToolCallProgress, progress.Phase) + s.Equal("Searching for flights", progress.State.Progress) + s.Equal(ToolCallInProgress, progress.State.Status) + _, again := s.apply(agent.ToolUpdated{ID: "call-1", Tool: "search_flights", Text: "Searching for flights"}) + s.False(again, "the same update changes nothing") +} + +func (s *ToolCallStateSuite) TestACallStoppedByItsContextFinishesCancelled() { + s.start("call-1", "search_flights") + + finished, ok := s.apply(agent.ToolRan{ID: "call-1", Tool: "search_flights", + Err: fmt.Errorf("session: search_flights stopped: %w", context.Canceled)}) + + s.Require().True(ok) + s.Equal(ToolCallCancelled, finished.State.Status) + s.True(finished.State.RejectedAt.IsZero(), "nobody declined it") +} + +func (s *ToolCallStateSuite) TestACallThatErrsFinishesFailedSayingWhy() { + s.start("call-1", "search_flights") + + finished, ok := s.apply(agent.ToolRan{ID: "call-1", Tool: "search_flights", Err: errors.New("the airline is down")}) + + s.Require().True(ok) + s.Equal(ToolCallFailed, finished.State.Status) + s.Equal("the airline is down", finished.State.Error) +} + +func (s *ToolCallStateSuite) TestAVoiceCallWaitsOnItsQuestionAndFinishesWithTheConfirmationThatRanIt() { + s.start("call-1", "transfer_money") + + waiting, ok := s.apply(agent.ToolAwaitingApproval{ID: "call-1", Tool: "transfer_money", Question: "Send 20 euros to Ana?"}) + s.Require().True(ok) + s.Equal(ToolCallWaiting, waiting.State.Status) + s.Equal("Send 20 euros to Ana?", waiting.State.Progress) + _, asked := s.apply(agent.ToolRan{ID: "call-1", Tool: "transfer_money", Result: "Ask the caller", Asked: true}) + s.False(asked, "the question is not the call's outcome") + _, confirmation := s.apply(agent.ToolStarted{ID: "call-2", Tool: agent.ConfirmTool, StartedAt: s.began}) + s.False(confirmation, "the confirmation is not a call of its own") + _, ok = s.apply(agent.ToolApprovalDecided{ID: "call-1", Allowed: true}) + s.Require().True(ok) + + finished, ok := s.apply(agent.ToolRan{ID: "call-2", Tool: agent.ConfirmTool, + Arguments: `{"tool_call_id":"call-1","allowed":true}`, Result: "sent"}) + + s.Require().True(ok) + s.Equal("call-1", finished.State.ID) + s.Equal(ToolCallSuccess, finished.State.Status) + s.False(finished.State.ApprovedAt.IsZero()) +} + +func (s *ToolCallStateSuite) TestAVoiceQuestionNobodyAnsweredInTimeIsRejected() { + s.start("call-1", "transfer_money") + s.apply(agent.ToolAwaitingApproval{ID: "call-1", Tool: "transfer_money", Question: "Send it?"}) + + lapsed, ok := s.apply(agent.ToolApprovalDecided{ID: "call-1", Summary: "Not confirmed in time"}) + + s.Require().True(ok) + s.Equal(ToolCallCancelled, lapsed.State.Status) + s.False(lapsed.State.RejectedAt.IsZero()) +} + +func (s *ToolCallStateSuite) TestAConnectorCallHeldBeforeItsStartCameThroughStartsWaiting() { + _, early := s.apply(agent.ToolAwaitingApproval{ID: "call-1", Tool: "github__merge", Question: "Allow github to run merge?"}) + s.False(early) + + started := s.start("call-1", "github__merge") + + s.Equal(ToolCallWaiting, started.State.Status) + s.Equal("Allow github to run merge?", started.State.Progress) +} diff --git a/acceleration/internal/session/tool_loading.go b/acceleration/internal/session/tool_loading.go new file mode 100644 index 0000000000..1706d24313 --- /dev/null +++ b/acceleration/internal/session/tool_loading.go @@ -0,0 +1,189 @@ +package session + +import ( + "cmp" + "errors" + "slices" + "strings" + + "github.com/GetStream/Vision-Agents/acceleration/internal/agent" + "github.com/GetStream/Vision-Agents/acceleration/internal/connectors/sources/mcp" + "github.com/GetStream/Vision-Agents/acceleration/internal/harness" + "github.com/GetStream/Vision-Agents/acceleration/internal/llm" + "github.com/GetStream/Vision-Agents/acceleration/internal/options" + "github.com/GetStream/Vision-Agents/acceleration/internal/store" +) + +// ErrToolNotConfigured is a function a session under an agent config was asked to offer that +// none of the config's tool groups names. +var ErrToolNotConfigured = errors.New("session: no tool group of the agent's config names the tool") + +// toolLoading is how a session offers each of its tools: a function an SDK provides by the +// config's tool group naming it, a connector's tool by its binding, and the router's own +// tools whole. +type toolLoading struct { + // configured is whether the session runs under an agent config, whose groups are then the + // whole of the functions it may offer. One opened without a config has nowhere to name + // them, and offers what its caller declares. + configured bool + functions map[string]store.ToolGroup + bindings map[string]store.ConnectorBinding +} + +func newToolLoading(spec Spec) toolLoading { + loading := toolLoading{ + configured: spec.ConfigID != "", + functions: map[string]store.ToolGroup{}, + bindings: map[string]store.ConnectorBinding{}, + } + for _, group := range spec.ToolGroups { + for _, name := range group.Tools { + loading.functions[strings.TrimSpace(name)] = group + } + } + for _, binding := range spec.ConnectorBindings { + loading.bindings[binding.Name] = binding + } + return loading +} + +// grouped reports whether the session may offer the function called name. Video frames are +// the session's own to ask a device for, whoever declares the tool. +func (l toolLoading) grouped(name string) bool { + if !l.configured || name == agent.VideoFramesTool { + return true + } + _, named := l.functions[name] + return named +} + +// ungrouped are the names among tools the session may not offer. +func (l toolLoading) ungrouped(tools []harness.Tool) []string { + var names []string + for _, tool := range tools { + if !l.grouped(tool.Name) { + names = append(names, tool.Name) + } + } + return names +} + +// missing are the functions the config's groups name that none of tools is, sorted: nothing +// on the session runs them. +func (l toolLoading) missing(tools []harness.Tool) []string { + var names []string + for name := range l.functions { + if !slices.ContainsFunc(tools, named(name)) { + names = append(names, name) + } + } + slices.Sort(names) + return names +} + +// mode is how the tool called name is offered: its group's loading, or its connector +// binding's, or full for everything else. +func (l toolLoading) mode(name string) string { + if group, named := l.functions[name]; named { + return cmp.Or(group.Loading, store.ToolLoadingFull) + } + if binding, bound := l.binding(name); bound { + return cmp.Or(binding.Loading, store.ToolLoadingFull) + } + return store.ToolLoadingFull +} + +// retention is how long what the tool called name was given and gave back is kept, as its +// group or connector binding says, before the policies shorten it. Empty is as long as the +// session. +func (l toolLoading) retention(name string) options.Retention { + if group, named := l.functions[name]; named { + return group.OutputRetention + } + if binding, bound := l.binding(name); bound { + return binding.OutputRetention + } + return "" +} + +// binding is the connector binding whose tool name is, offered as __. +func (l toolLoading) binding(name string) (store.ConnectorBinding, bool) { + alias, _, cut := strings.Cut(name, mcp.Separator) + if !cut { + return store.ConnectorBinding{}, false + } + binding, bound := l.bindings[alias] + return binding, bound +} + +// summarises reports whether any tool the session may offer is offered by a summary. +func (l toolLoading) summarises() bool { + for _, group := range l.functions { + if group.Loading == store.ToolLoadingSummary { + return true + } + } + for _, binding := range l.bindings { + if binding.Loading == store.ToolLoadingSummary { + return true + } + } + return false +} + +// arrange is tools as the model is offered them, in the order given: a summarised one by its +// first line and the shape of its arguments, a full one whole, and a deferred one held back +// for load_tool. +func (l toolLoading) arrange(tools []harness.Tool) harness.Tools { + var arranged harness.Tools + for _, tool := range tools { + switch l.mode(tool.Name) { + case store.ToolLoadingSummary: + arranged.Tools = append(arranged.Tools, briefTool(tool)) + case store.ToolLoadingDeferred: + arranged.Deferred = append(arranged.Deferred, tool) + default: + arranged.Tools = append(arranged.Tools, tool) + } + } + return arranged +} + +// ToolLoading is how the session offers the tool called name: store.ToolLoadingFull, +// ToolLoadingSummary or ToolLoadingDeferred. +func (s *Session) ToolLoading(name string) string { return s.loading.mode(name) } + +// describeOffer adds to a call's payload how the tool called name is offered and what its +// definition costs every turn that carries it, whole and as a summary: what choosing its +// loading weighs. A tool the session does not describe, such as a built-in, adds nothing. +func (s *Session) describeOffer(payload map[string]any, name string) { + all := slices.Concat(s.declaredTools(), s.otherTools) + at := slices.IndexFunc(all, named(name)) + if at < 0 { + return + } + mode := s.loading.mode(name) + payload["loading"] = mode + payload["definition_tokens"] = definitionTokens(all[at]) + if mode == store.ToolLoadingSummary { + payload["summary_tokens"] = definitionTokens(briefTool(all[at])) + } +} + +func definitionTokens(tool harness.Tool) int64 { + return llm.ToolTokens(llm.Tool{Name: tool.Name, Description: tool.Description, Parameters: tool.Parameters}) +} + +// summarised is the tool called name as its server or SDK describes it, when it is one the +// session offers by a summary. +func (s *Session) summarised(name string) (harness.Tool, bool) { + if s.loading.mode(name) != store.ToolLoadingSummary { + return harness.Tool{}, false + } + all := slices.Concat(s.declaredTools(), s.otherTools) + at := slices.IndexFunc(all, named(name)) + if at < 0 { + return harness.Tool{}, false + } + return all[at], true +} diff --git a/acceleration/internal/session/tool_policy_test.go b/acceleration/internal/session/tool_policy_test.go index d6d8be952d..021fa2d8b2 100644 --- a/acceleration/internal/session/tool_policy_test.go +++ b/acceleration/internal/session/tool_policy_test.go @@ -66,7 +66,8 @@ func (s *ToolPolicySuite) TestAWorkersToolKeepsItsPolicies() { {Name: "investigate_sdk", Description: "Read SDK source", Interruptible: true, Timeout: time.Second}, }}} - tools, _ := m.hostedTools(Spec{CustomerID: "acme", AgentID: "support"}, "session-1", nil, nil, nil) + spec := Spec{CustomerID: "acme", AgentID: "support"} + tools, _ := m.hostedTools(spec, newToolLoading(spec), "session-1", nil, nil, nil) s.Require().Len(tools, 1) s.True(tools[0].Interruptible) @@ -79,7 +80,8 @@ func (s *ToolPolicySuite) TestAWorkersToolAsksItsApprovalQuestion() { {Name: "search_flights", Description: "Search flights"}, }}} - tools, _ := m.hostedTools(Spec{CustomerID: "acme", AgentID: "support"}, "session-1", nil, nil, nil) + spec := Spec{CustomerID: "acme", AgentID: "support"} + tools, _ := m.hostedTools(spec, newToolLoading(spec), "session-1", nil, nil, nil) s.Require().Len(tools, 2) s.Equal(&harness.ToolApproval{Title: "Book flight {flight_id}?"}, tools[0].Approval) diff --git a/acceleration/internal/store/agents.go b/acceleration/internal/store/agents.go index 34f50846f4..65d8604e10 100644 --- a/acceleration/internal/store/agents.go +++ b/acceleration/internal/store/agents.go @@ -75,7 +75,7 @@ var configColumns = []string{ "name", "role", "mode", "region", "stt", "tts", "sts", "voice", "llm", "llm_fallbacks", "agents", "subagent", "video_source", "video_max_frames", "search", "image", "instructions", "greeting", "greeting_mode", "guardrail", "review_instructions", "ui_instructions", "skills", "connectors", "left_out_connectors", "channels", "chat_channels", "scheduling", "calls", "keyterms", "visible_tools", "knowledge_namespace", "sandbox", "sandbox_options", "harness", "tags", - "dispatch_incoming_call", "dispatch_text", "episode_cards", "episode_cards_record", "progressive_tools", "tool_output_retention", "resume_history", "floor_control", "sync_hash", "updated_at", + "dispatch_incoming_call", "dispatch_text", "episode_cards", "episode_cards_record", "tool_groups", "resume_history", "floor_control", "sync_hash", "updated_at", } // UpdateAgentConfig replaces a config a customer holds. Every field is written, so an diff --git a/acceleration/internal/store/models.go b/acceleration/internal/store/models.go index c300d12419..61d8b82a88 100644 --- a/acceleration/internal/store/models.go +++ b/acceleration/internal/store/models.go @@ -222,6 +222,11 @@ type ToolUsage struct { Tool string `bun:"tool"` Calls int64 `bun:"calls"` Failed int64 `bun:"failed"` + // Sessions is how many sessions made at least one of the calls. + Sessions int64 `bun:"sessions"` + // AvgTurnsBeforeUse is how many turns a session took before its first call, nil on the + // row over every tool. + AvgTurnsBeforeUse *float64 `bun:"avg_turns_before_use"` // AvgTokens is what a call put in front of the model, its arguments and result, // estimated the way a prompt's composition counts tool use. AvgTokens float64 `bun:"avg_tokens"` @@ -229,6 +234,18 @@ type ToolUsage struct { // group has both. AvgMs *float64 `bun:"avg_ms"` P99Ms *float64 `bun:"p99_ms"` + // Loading, DefinitionTokens and SummaryTokens are how the latest call that recorded them + // had the tool offered, and what its definition costs a turn whole and as a summary. + // Empty and nil when no call did; SummaryTokens is set only for a summarised tool. + Loading string `bun:"loading"` + DefinitionTokens *int64 `bun:"definition_tokens"` + SummaryTokens *int64 `bun:"summary_tokens"` +} + +// SessionTurns is how many sessions took a turn and how many turns they took. +type SessionTurns struct { + Sessions int64 `bun:"sessions"` + Turns int64 `bun:"turns"` } // TurnSample is what a finished turn adds to its hour's stats. The legs are pointers @@ -635,16 +652,12 @@ type AgentConfig struct { // (20261217090000_agent_config_episode_cards_record.sql): the record mode. Read only beside // EpisodeCards off; WritesCards is the two together. EpisodeCardsRecord bool `bun:"episode_cards_record,notnull"` - // ProgressiveTools offers plugin, MCP server and connector tools by a summary, and - // sends a tool's full description the first time it is called instead of running it - // (20261008130000_agent_config_progressive_tools.sql). Off unless a config turns it on. - ProgressiveTools bool `bun:"progressive_tools,notnull"` - // ToolOutputRetention is how long the config's sessions keep what their tools were - // given and gave back on the turns they record: options.RetentionNone for not at all, a - // duration such as 30d, or empty for as long as the session. The app's and the - // organization's policies can shorten it and never lengthen it - // (20261113120000_tool_output_retention.sql). - ToolOutputRetention options.Retention `bun:"tool_output_retention,notnull"` + // ToolGroups are the functions an SDK provides that the config's sessions may offer, and + // how each group is loaded (20270102090000_agent_config_tool_groups.sql). A function no + // group names is not offered. The row's progressive_tools and tool_output_retention + // columns, which groups and connector bindings replaced, are not modelled: a router still + // running the release before reads them, so they are left in place. + ToolGroups []ToolGroup `bun:"tool_groups,type:jsonb"` // ResumeHistory is how the config's sessions are handed back a conversation they // carry on. The zero value is the router's default. ResumeHistory ResumeHistory `bun:"resume_history,type:jsonb,notnull"` @@ -721,6 +734,34 @@ type ConnectorBinding struct { // allow each one before it is sent. The field is JSON inside agent_configs.connectors, so // an older router drops it and sends those calls with nobody asked. RequireApproval []string `json:"require_approval,omitempty"` + // Loading is how the binding's tools are offered: ToolLoadingFull, ToolLoadingSummary or + // ToolLoadingDeferred. Empty is full. + Loading string `json:"loading,omitempty"` + // OutputRetention is how long the turns a session records keep what the binding's tools + // were given and gave back: options.RetentionNone, a duration, or empty for as long as the + // session. The app's and the organization's policies can shorten it. + OutputRetention options.Retention `json:"output_retention,omitempty"` +} + +// The ways a tool is offered. Full sends its whole description and input schema from the +// start; summary sends the first line of its description and a schema without descriptions, +// and the first call returns the rest; deferred offers nothing until the model finds the tool +// with search_tool and loads it with load_tool. +const ( + ToolLoadingFull = "full" + ToolLoadingSummary = "summary" + ToolLoadingDeferred = "deferred" +) + +// ToolGroup is a named set of functions an SDK provides, as agent_configs.tool_groups stores it. +type ToolGroup struct { + Name string `json:"name"` + // Loading is ToolLoadingFull, ToolLoadingSummary or ToolLoadingDeferred. Empty is full. + Loading string `json:"loading,omitempty"` + // Tools are the functions' names. + Tools []string `json:"tools"` + // OutputRetention is ConnectorBinding.OutputRetention for the group's functions. + OutputRetention options.Retention `json:"output_retention,omitempty"` } // RequiresApproval reports whether the binding's require_approval names tool, by its name at @@ -2276,9 +2317,11 @@ type AgentResponseItem struct { } // redactedToolKeys are what a redacted tool item keeps of its payload: what ties a call to -// its result and says which product and SDK ran it. Anything else the payload carries, -// the arguments above all, goes. An error is kept as having happened, not as its text. -var redactedToolKeys = []string{"call_id", "product", "sdk"} +// its result, says which product and SDK ran it, and the counts tool usage reads: how the +// tool was offered, what its definition and the call cost in tokens. Anything else the +// payload carries, the arguments above all, goes. An error is kept as having happened, not +// as its text. +var redactedToolKeys = []string{"call_id", "product", "sdk", "loading", "definition_tokens", "summary_tokens", "tokens"} // Redact takes out what a tool item carried of the call itself: its text (the result, or // a search's query), its arguments, and the wording of its error. What is left still says diff --git a/acceleration/internal/store/sessions.go b/acceleration/internal/store/sessions.go index c2027b8587..94d4dfe628 100644 --- a/acceleration/internal/store/sessions.go +++ b/acceleration/internal/store/sessions.go @@ -769,6 +769,10 @@ UPDATE agent_response_items AS t SET 'call_id', t.payload->'call_id', 'product', t.payload->'product', 'sdk', t.payload->'sdk', + 'loading', t.payload->'loading', + 'definition_tokens', t.payload->'definition_tokens', + 'summary_tokens', t.payload->'summary_tokens', + 'tokens', t.payload->'tokens', 'error', CASE WHEN t.payload->'error' IS NOT NULL THEN to_jsonb('redacted'::text) END, 'redacted', true)), redact_at = NULL diff --git a/acceleration/internal/store/sessions_test.go b/acceleration/internal/store/sessions_test.go index 7f31c5b98f..41df9df278 100644 --- a/acceleration/internal/store/sessions_test.go +++ b/acceleration/internal/store/sessions_test.go @@ -1072,7 +1072,7 @@ func (s *StoreSuite) TestARedactedToolCallStillCountsInToolUsage() { _, err := s.store.RedactToolOutput(s.ctx, due, 100) s.Require().NoError(err) - overall, tools, err := s.store.CustomerToolUsage(s.ctx, "app", s.base, s.base.Add(24*time.Hour)) + overall, tools, err := s.store.CustomerToolUsage(s.ctx, "app", "", s.base, s.base.Add(24*time.Hour)) s.Require().NoError(err) s.EqualValues(1, overall.Calls) s.EqualValues(1, overall.Failed) diff --git a/acceleration/internal/store/store.go b/acceleration/internal/store/store.go index e2cc007e80..4a48acbfc2 100644 --- a/acceleration/internal/store/store.go +++ b/acceleration/internal/store/store.go @@ -1431,55 +1431,106 @@ func (s *Store) peakAcrossShards(ctx context.Context, spans, customerID string, } // CustomerToolUsage returns how every tool one customer's agents ran in a window was used, -// most called first, and the same figures over all of them. A web search the agent made -// has no result item, so it is not a tool call here. -func (s *Store) CustomerToolUsage(ctx context.Context, customerID string, from, to time.Time) (ToolUsage, []ToolUsage, error) { +// most called first, and the same figures over all of them. A configID keeps to the +// sessions of that agent config, and empty is every one. A web search the agent made has no +// result item, so it is not a tool call here. +// +// A turn is a top-level response that was not rewound, and a call made in a delegated task +// belongs to the turn that delegated it. +func (s *Store) CustomerToolUsage(ctx context.Context, customerID, configID string, from, to time.Time) (ToolUsage, []ToolUsage, error) { if customerID == "" { return ToolUsage{}, nil, stack.Wrap(errors.New("store: customer id is required")) } type usageRow struct { - Tool string `bun:"tool"` - Overall bool `bun:"overall"` - Calls int64 `bun:"calls"` - Failed int64 `bun:"failed"` - AvgTokens float64 `bun:"avg_tokens"` - AvgMs *float64 `bun:"avg_ms"` - P99Ms *float64 `bun:"p99_ms"` + Tool string `bun:"tool"` + Overall bool `bun:"overall"` + Calls int64 `bun:"calls"` + Failed int64 `bun:"failed"` + Sessions int64 `bun:"sessions"` + Firsts int64 `bun:"firsts"` + AvgTurnsBeforeUse *float64 `bun:"avg_turns_before_use"` + AvgTokens float64 `bun:"avg_tokens"` + AvgMs *float64 `bun:"avg_ms"` + P99Ms *float64 `bun:"p99_ms"` + OfferedAt *time.Time `bun:"offered_at"` + Loading string `bun:"loading"` + DefinitionTokens *int64 `bun:"definition_tokens"` + SummaryTokens *int64 `bun:"summary_tokens"` } lists, err := everyShard(ctx, s, func(ctx context.Context, db bun.IDB) ([]usageRow, error) { var rows []usageRow err := db.NewRaw(` -WITH ran AS ( +WITH turns AS ( + SELECT + a.id, + ROW_NUMBER() OVER (PARTITION BY a.session_id ORDER BY a.created_at, a.id) - 1 AS before + FROM agent_responses AS a + JOIN agent_sessions AS s ON s.id = a.session_id + WHERE a.customer_id = ?3 AND (?6 = '' OR s.config_id = ?6) AND a.created_at < ?5 + AND a.parent_response_id IS NULL AND a.rewound_at IS NULL +), +ran AS ( SELECT r.tool_name AS tool, + a.session_id, + t.before, + r.at, r.payload->>'error' IS NOT NULL AS failed, - (octet_length(r.tool_name) + octet_length(COALESCE(r.payload->>'arguments', '')) - + octet_length(r.text) + ?0 - 1) / ?0 AS tokens, - EXTRACT(EPOCH FROM (r.at - started.at)) * 1000 AS ms + COALESCE((r.payload->>'tokens')::double precision, + (octet_length(r.tool_name) + octet_length(COALESCE(r.payload->>'arguments', '')) + + octet_length(r.text) + ?0 - 1) / ?0) AS tokens, + EXTRACT(EPOCH FROM (r.at - started.at)) * 1000 AS ms, + started.payload->>'loading' AS loading, + (started.payload->>'definition_tokens')::bigint AS definition_tokens, + (started.payload->>'summary_tokens')::bigint AS summary_tokens FROM agent_responses AS a + JOIN agent_sessions AS s ON s.id = a.session_id JOIN agent_response_items AS r ON r.response_id = a.id AND r.kind = ?1 + LEFT JOIN turns AS t ON t.id = COALESCE(a.parent_response_id, a.id) LEFT JOIN LATERAL ( - SELECT c.at FROM agent_response_items AS c + SELECT c.at, c.payload FROM agent_response_items AS c WHERE c.response_id = r.response_id AND c.kind = ?2 AND c.tool_name = r.tool_name AND c.ordinal < r.ordinal AND COALESCE(c.payload->>'call_id', '') = COALESCE(r.payload->>'call_id', '') ORDER BY c.ordinal DESC LIMIT 1 ) AS started ON true - WHERE a.customer_id = ?3 AND a.created_at < ?5 AND r.at >= ?4 AND r.at < ?5 + WHERE a.customer_id = ?3 AND (?6 = '' OR s.config_id = ?6) AND a.created_at < ?5 + AND r.at >= ?4 AND r.at < ?5 +), +firsts AS ( + SELECT tool, COUNT(*) AS firsts, AVG(before)::double precision AS avg_turns_before_use + FROM (SELECT tool, MIN(before) AS before FROM ran WHERE before IS NOT NULL GROUP BY tool, session_id) AS first + GROUP BY tool +), +offered AS ( + SELECT DISTINCT ON (tool) tool, at, loading, definition_tokens, summary_tokens + FROM ran WHERE definition_tokens IS NOT NULL + ORDER BY tool, at DESC ) SELECT - COALESCE(tool, '') AS tool, - GROUPING(tool) = 1 AS overall, + COALESCE(ran.tool, '') AS tool, + GROUPING(ran.tool) = 1 AS overall, COUNT(*) AS calls, COUNT(*) FILTER (WHERE failed) AS failed, + COUNT(DISTINCT ran.session_id) AS sessions, + COALESCE(f.firsts, 0) AS firsts, + f.avg_turns_before_use, COALESCE(AVG(tokens), 0)::double precision AS avg_tokens, AVG(ms)::double precision AS avg_ms, - percentile_cont(0.99) WITHIN GROUP (ORDER BY ms) AS p99_ms + percentile_cont(0.99) WITHIN GROUP (ORDER BY ms) AS p99_ms, + o.at AS offered_at, + COALESCE(o.loading, '') AS loading, + o.definition_tokens, + o.summary_tokens FROM ran -GROUP BY GROUPING SETS ((tool), ()) +LEFT JOIN firsts AS f ON f.tool = ran.tool +LEFT JOIN offered AS o ON o.tool = ran.tool +GROUP BY GROUPING SETS ( + (ran.tool, f.firsts, f.avg_turns_before_use, o.at, o.loading, o.definition_tokens, o.summary_tokens), + ()) ORDER BY overall DESC, calls DESC, tool ASC`, - charsPerToken, ItemToolResult, ItemToolCall, customerID, from, to, + charsPerToken, ItemToolResult, ItemToolCall, customerID, from, to, configID, ).Scan(ctx, &rows) return rows, err }) @@ -1496,6 +1547,9 @@ ORDER BY overall DESC, calls DESC, tool ASC`, var order []usageKey sums := map[usageKey]*ToolUsage{} tokens, ms, p99s, calls := map[usageKey][]*float64{}, map[usageKey][]*float64{}, map[usageKey][]*float64{}, map[usageKey][]int64{} + befores, firsts := map[usageKey][]*float64{}, map[usageKey][]int64{} + // The definition is the latest call's on any shard: a tool's description can change. + offeredAt := map[usageKey]time.Time{} for _, rows := range lists { for _, row := range rows { k := usageKey{row.Tool, row.Overall} @@ -1505,11 +1559,18 @@ ORDER BY overall DESC, calls DESC, tool ASC`, } sums[k].Calls += row.Calls sums[k].Failed += row.Failed + sums[k].Sessions += row.Sessions avgTokens := row.AvgTokens tokens[k] = append(tokens[k], &avgTokens) ms[k] = append(ms[k], row.AvgMs) p99s[k] = append(p99s[k], row.P99Ms) calls[k] = append(calls[k], row.Calls) + befores[k] = append(befores[k], row.AvgTurnsBeforeUse) + firsts[k] = append(firsts[k], row.Firsts) + if row.OfferedAt != nil && row.OfferedAt.After(offeredAt[k]) { + offeredAt[k] = *row.OfferedAt + sums[k].Loading, sums[k].DefinitionTokens, sums[k].SummaryTokens = row.Loading, row.DefinitionTokens, row.SummaryTokens + } } } var overall ToolUsage @@ -1520,6 +1581,7 @@ ORDER BY overall DESC, calls DESC, tool ASC`, usage.AvgTokens = *avg } usage.AvgMs, usage.P99Ms = weighted(ms[k], calls[k]), weighted(p99s[k], calls[k]) + usage.AvgTurnsBeforeUse = weighted(befores[k], firsts[k]) if len(lists) == 1 { usage.AvgMs, usage.P99Ms = ms[k][0], p99s[k][0] } @@ -1535,6 +1597,37 @@ ORDER BY overall DESC, calls DESC, tool ASC`, return overall, tools, nil } +// CustomerTurns counts the sessions one customer's agents took a turn in during a window, +// and those turns, as CustomerToolUsage counts a turn. A configID keeps to the sessions of +// that agent config, and empty is every one. +func (s *Store) CustomerTurns(ctx context.Context, customerID, configID string, from, to time.Time) (SessionTurns, error) { + if customerID == "" { + return SessionTurns{}, stack.Wrap(errors.New("store: customer id is required")) + } + counts, err := everyShard(ctx, s, func(ctx context.Context, db bun.IDB) (SessionTurns, error) { + var counts SessionTurns + err := db.NewRaw(` +SELECT COUNT(DISTINCT a.session_id) AS sessions, COUNT(*) AS turns +FROM agent_responses AS a +JOIN agent_sessions AS s ON s.id = a.session_id +WHERE a.customer_id = ?0 AND (?1 = '' OR s.config_id = ?1) + AND a.created_at >= ?2 AND a.created_at < ?3 + AND a.parent_response_id IS NULL AND a.rewound_at IS NULL`, + customerID, configID, from, to, + ).Scan(ctx, &counts) + return counts, err + }) + if err != nil { + return SessionTurns{}, stack.Wrap(fmt.Errorf("store: turns: %w", err)) + } + var total SessionTurns + for _, shard := range counts { + total.Sessions += shard.Sessions + total.Turns += shard.Turns + } + return total, nil +} + // charsPerToken is the rule of thumb llm.Compose estimates tool use with. const charsPerToken = 4 diff --git a/acceleration/internal/store/tool_groups_migration_test.go b/acceleration/internal/store/tool_groups_migration_test.go new file mode 100644 index 0000000000..2b880f02c7 --- /dev/null +++ b/acceleration/internal/store/tool_groups_migration_test.go @@ -0,0 +1,118 @@ +//go:build integration + +package store + +import ( + "context" + "os" + "strings" + "testing" + + "github.com/pressly/goose/v3" + "github.com/stretchr/testify/suite" + + "github.com/GetStream/Vision-Agents/acceleration/internal/testenv" + "github.com/GetStream/Vision-Agents/acceleration/migrations" +) + +// Versions either side of 20270102090000_agent_config_tool_groups.sql. +const ( + schemaBeforeToolGroups int64 = 20270101090000 + toolGroupsVersion int64 = 20270102090000 +) + +// ToolGroupsMigrationSuite stops a database of its own short of tool groups, writes configs +// with the agent-wide tool settings, then applies the migration. +type ToolGroupsMigrationSuite struct { + suite.Suite + dsn string + ctx context.Context + db *Store +} + +func TestToolGroupsMigrationSuite(t *testing.T) { + suite.Run(t, new(ToolGroupsMigrationSuite)) +} + +func (s *ToolGroupsMigrationSuite) SetupSuite() { + dsn := os.Getenv(DSNEnvVar) + if dsn == "" { + s.T().Skipf("%s not set", DSNEnvVar) + } + s.dsn = testenv.Database(dsn, "toolgroupsmig") + s.ctx = context.Background() +} + +func (s *ToolGroupsMigrationSuite) SetupTest() { + db, err := Open(s.dsn) + s.Require().NoError(err) + s.T().Cleanup(func() { s.NoError(db.Close()) }) + var name string + s.Require().NoError(db.DB().QueryRowContext(s.ctx, "SELECT current_database()").Scan(&name)) + s.Require().True(strings.HasSuffix(name, "_test"), "refusing to drop the schema of %s, which is not a test database", name) + s.Require().Contains(name, "toolgroupsmig") + s.exec(db, "DROP SCHEMA public CASCADE; CREATE SCHEMA public") + goose.SetBaseFS(migrations.FS) + s.Require().NoError(goose.SetDialect("postgres")) + s.Require().NoError(goose.UpToContext(s.ctx, db.DB().DB, ".", schemaBeforeToolGroups)) + s.db = db +} + +// connectors is the stored connector bindings of the config called id, read by its column +// rather than through the model, which carries columns later migrations add. +func (s *ToolGroupsMigrationSuite) connectors(id string) string { + var connectors string + s.Require().NoError(s.db.DB().QueryRowContext(s.ctx, + `SELECT connectors::text FROM agent_configs WHERE id = ?`, id).Scan(&connectors)) + return connectors +} + +func (s *ToolGroupsMigrationSuite) TestAProgressiveConfigsConnectorsAreSummarisedAndKeepTheirRetention() { + s.exec(s.db, `INSERT INTO agent_configs (id, customer_id, name, progressive_tools, tool_output_retention, connectors) +VALUES ('older', 'app', 'support', true, '7d', + '[{"name": "linear", "connector_id": "c1"}, {"name": "sentry", "connector_id": "c2", "output_retention": "none"}]')`) + + s.Require().NoError(goose.UpToContext(s.ctx, s.db.DB().DB, ".", toolGroupsVersion)) + + s.JSONEq(`[ + {"name": "linear", "connector_id": "c1", "loading": "summary", "output_retention": "7d"}, + {"name": "sentry", "connector_id": "c2", "loading": "summary", "output_retention": "none"} + ]`, s.connectors("older"), "a binding that says its own retention keeps it, and the order stays") +} + +func (s *ToolGroupsMigrationSuite) TestAConfigWithoutTheSettingsIsLeftAsItWas() { + s.exec(s.db, `INSERT INTO agent_configs (id, customer_id, name, connectors) +VALUES ('plain', 'app', 'support', '[{"name": "linear", "connector_id": "c1"}]')`) + + s.Require().NoError(goose.UpToContext(s.ctx, s.db.DB().DB, ".", toolGroupsVersion)) + + s.JSONEq(`[{"name": "linear", "connector_id": "c1"}]`, s.connectors("plain")) + var groups *string + s.Require().NoError(s.db.DB().QueryRowContext(s.ctx, + `SELECT tool_groups::text FROM agent_configs WHERE id = 'plain'`).Scan(&groups)) + s.Nil(groups, "a config from before names no function") +} + +func (s *ToolGroupsMigrationSuite) TestAMigrationThatStoppedHalfwayRunsAgain() { + s.exec(s.db, `ALTER TABLE agent_configs ADD COLUMN tool_groups JSONB`) + + s.Require().NoError(goose.UpToContext(s.ctx, s.db.DB().DB, ".", toolGroupsVersion)) +} + +func (s *ToolGroupsMigrationSuite) TestTheMigrationComesOffAgain() { + s.Require().NoError(goose.UpToContext(s.ctx, s.db.DB().DB, ".", toolGroupsVersion)) + + s.Require().NoError(goose.DownToContext(s.ctx, s.db.DB().DB, ".", schemaBeforeToolGroups)) + + var columns int + s.Require().NoError(s.db.DB().QueryRowContext(s.ctx, `SELECT count(*) FROM information_schema.columns +WHERE table_name = 'agent_configs' AND column_name = 'tool_groups'`).Scan(&columns)) + s.Zero(columns) +} + +func (s *ToolGroupsMigrationSuite) exec(db *Store, statements ...string) { + for _, statement := range statements { + _, err := db.DB().ExecContext(s.ctx, statement) + s.Require().NoError(err, statement) + } +} diff --git a/acceleration/internal/store/usage_test.go b/acceleration/internal/store/usage_test.go index 6a75e9e2e3..3d1ae09140 100644 --- a/acceleration/internal/store/usage_test.go +++ b/acceleration/internal/store/usage_test.go @@ -379,7 +379,7 @@ func (s *StoreSuite) TestToolUsageIsMostCalledFirstWithHowLongEachTook() { s.ranTool(session, "lookup_order", s.base.Add(2*time.Minute), 300*time.Millisecond, "shipped", false) s.ranTool(session, "refund", s.base.Add(3*time.Minute), 2*time.Second, "", true) - overall, tools, err := s.store.CustomerToolUsage(s.ctx, "acme", s.base, s.base.Add(24*time.Hour)) + overall, tools, err := s.store.CustomerToolUsage(s.ctx, "acme", "", s.base, s.base.Add(24*time.Hour)) s.Require().NoError(err) s.Require().Len(tools, 2) @@ -405,13 +405,149 @@ func (s *StoreSuite) TestToolUsageIsOnlyTheCallingCustomers() { theirs := s.opened("theirs", "globex", s.base, nil) s.ranTool(theirs, "lookup_order", s.base.Add(time.Minute), time.Second, "shipped", false) - overall, tools, err := s.store.CustomerToolUsage(s.ctx, "acme", s.base, s.base.Add(24*time.Hour)) + overall, tools, err := s.store.CustomerToolUsage(s.ctx, "acme", "", s.base, s.base.Add(24*time.Hour)) s.Require().NoError(err) s.Empty(tools) s.Zero(overall.Calls) s.Nil(overall.AvgMs) } +// turned records a turn the session took at, before change. +func (s *StoreSuite) turned(session *AgentSession, id string, at time.Time, change func(*AgentResponse)) { + response := &AgentResponse{ID: id, SessionID: session.ID, CustomerID: session.CustomerID, CreatedAt: at} + if change != nil { + change(response) + } + s.Require().NoError(s.store.StartResponse(s.ctx, response)) +} + +// calledTool records one call of tool in the turn responseID, its result a second later, with +// what the session put on each item. +func (s *StoreSuite) calledTool(session *AgentSession, responseID, tool string, at time.Time, call, result map[string]any) { + s.Require().NoError(s.store.AppendResponseItems(s.ctx, []AgentResponseItem{ + {ResponseID: responseID, Ordinal: 0, SessionID: session.ID, Kind: ItemToolCall, ToolName: tool, + Payload: call, At: at}, + {ResponseID: responseID, Ordinal: 1, SessionID: session.ID, Kind: ItemToolResult, ToolName: tool, + Text: "shipped", Payload: result, At: at.Add(time.Second)}, + })) +} + +func (s *StoreSuite) TestToolUsageOfOneAgentIsOutOfThatAgentsSessions() { + for i := range 4 { + session := s.opened(fmt.Sprintf("a%d", i), "acme", s.base, func(session *AgentSession) { session.ConfigID = "cfg-a" }) + s.turned(session, session.ID+"-1", s.base.Add(time.Minute), nil) + if i == 0 { + s.calledTool(session, session.ID+"-1", "lookup_order", s.base.Add(time.Minute), + map[string]any{"call_id": "c1"}, map[string]any{"call_id": "c1"}) + } + } + other := s.opened("b0", "acme", s.base, func(session *AgentSession) { session.ConfigID = "cfg-b" }) + s.turned(other, "b0-1", s.base.Add(time.Minute), nil) + s.calledTool(other, "b0-1", "refund", s.base.Add(time.Minute), map[string]any{"call_id": "c1"}, map[string]any{"call_id": "c1"}) + + turns, err := s.store.CustomerTurns(s.ctx, "acme", "cfg-a", s.base, s.base.Add(24*time.Hour)) + s.Require().NoError(err) + s.Equal(SessionTurns{Sessions: 4, Turns: 4}, turns) + overall, tools, err := s.store.CustomerToolUsage(s.ctx, "acme", "cfg-a", s.base, s.base.Add(24*time.Hour)) + s.Require().NoError(err) + s.Require().Len(tools, 1, "the other agent's refund is not this one's") + s.Equal("lookup_order", tools[0].Tool) + s.EqualValues(1, tools[0].Sessions) + s.EqualValues(1, overall.Sessions) + + _, tools, err = s.store.CustomerToolUsage(s.ctx, "acme", "cfg-b", s.base, s.base.Add(24*time.Hour)) + s.Require().NoError(err) + s.Require().Len(tools, 1) + s.Equal("refund", tools[0].Tool) + + everyone, err := s.store.CustomerTurns(s.ctx, "acme", "", s.base, s.base.Add(24*time.Hour)) + s.Require().NoError(err) + s.EqualValues(5, everyone.Sessions) +} + +func (s *StoreSuite) TestTurnsBeforeAToolIsUsedAreTheConversationsOwn() { + first := s.opened("first", "acme", s.base, nil) + s.turned(first, "first-1", s.base, nil) + rewound := s.base.Add(2 * time.Minute) + s.turned(first, "first-gone", s.base.Add(time.Minute), func(response *AgentResponse) { response.RewoundAt = &rewound }) + s.turned(first, "first-2", s.base.Add(2*time.Minute), nil) + s.turned(first, "first-task", s.base.Add(3*time.Minute), func(response *AgentResponse) { + response.ParentResponseID = "first-2" + }) + s.turned(first, "first-3", s.base.Add(4*time.Minute), nil) + s.calledTool(first, "first-3", "lookup_order", s.base.Add(4*time.Minute), map[string]any{"call_id": "c1"}, map[string]any{"call_id": "c1"}) + s.turned(first, "first-4", s.base.Add(5*time.Minute), nil) + s.calledTool(first, "first-4", "lookup_order", s.base.Add(5*time.Minute), map[string]any{"call_id": "c2"}, map[string]any{"call_id": "c2"}) + + second := s.opened("second", "acme", s.base, nil) + s.turned(second, "second-1", s.base, nil) + s.turned(second, "second-task", s.base.Add(time.Minute), func(response *AgentResponse) { + response.ParentResponseID = "second-1" + }) + s.calledTool(second, "second-task", "lookup_order", s.base.Add(time.Minute), map[string]any{"call_id": "c1"}, map[string]any{"call_id": "c1"}) + + overall, tools, err := s.store.CustomerToolUsage(s.ctx, "acme", "", s.base, s.base.Add(24*time.Hour)) + s.Require().NoError(err) + s.Require().Len(tools, 1) + s.EqualValues(3, tools[0].Calls) + s.EqualValues(2, tools[0].Sessions) + s.Require().NotNil(tools[0].AvgTurnsBeforeUse) + // Two turns before the first session's first call, none before the second's: a call in a + // delegated task is the turn that delegated it's. + s.InDelta(1.0, *tools[0].AvgTurnsBeforeUse, 0.001) + s.Nil(overall.AvgTurnsBeforeUse) + + turns, err := s.store.CustomerTurns(s.ctx, "acme", "", s.base, s.base.Add(24*time.Hour)) + s.Require().NoError(err) + s.Equal(SessionTurns{Sessions: 2, Turns: 5}, turns) +} + +func (s *StoreSuite) TestToolUsageReportsTheLatestDefinitionAToolWasOfferedWith() { + session := s.opened("one", "acme", s.base, nil) + s.turned(session, "one-1", s.base.Add(time.Minute), nil) + s.calledTool(session, "one-1", "lookup_order", s.base.Add(time.Minute), + map[string]any{"call_id": "c1", "loading": ToolLoadingFull, "definition_tokens": 461}, map[string]any{"call_id": "c1"}) + s.turned(session, "one-2", s.base.Add(2*time.Minute), nil) + s.calledTool(session, "one-2", "lookup_order", s.base.Add(2*time.Minute), + map[string]any{"call_id": "c2", "loading": ToolLoadingSummary, "definition_tokens": 470, "summary_tokens": 84}, + map[string]any{"call_id": "c2"}) + s.turned(session, "one-3", s.base.Add(3*time.Minute), nil) + s.calledTool(session, "one-3", "lookup_order", s.base.Add(3*time.Minute), map[string]any{"call_id": "c3"}, map[string]any{"call_id": "c3"}) + + _, tools, err := s.store.CustomerToolUsage(s.ctx, "acme", "", s.base, s.base.Add(24*time.Hour)) + s.Require().NoError(err) + s.Require().Len(tools, 1) + s.Equal(ToolLoadingSummary, tools[0].Loading, "the config moved it to a summary") + s.Require().NotNil(tools[0].DefinitionTokens) + s.EqualValues(470, *tools[0].DefinitionTokens) + s.Require().NotNil(tools[0].SummaryTokens) + s.EqualValues(84, *tools[0].SummaryTokens) +} + +func (s *StoreSuite) TestARedactedCallStillReportsWhatItAndItsToolCost() { + session := s.opened("one", "acme", s.base, nil) + s.turned(session, "one-1", s.base, nil) + s.calledTool(session, "one-1", "lookup_order", s.base, + map[string]any{"call_id": "c1", "loading": ToolLoadingSummary, "definition_tokens": 461, "summary_tokens": 84}, + map[string]any{"call_id": "c1", "arguments": `{"order":"A-1"}`, "tokens": 250}) + _, err := s.store.db.NewUpdate().Model((*AgentResponseItem)(nil)). + Set("redact_at = ?", s.base).Where("session_id = ?", "one").Exec(s.ctx) + s.Require().NoError(err) + redacted, err := s.store.RedactToolOutput(s.ctx, s.base.Add(time.Hour), 100) + s.Require().NoError(err) + s.EqualValues(2, redacted) + + _, tools, err := s.store.CustomerToolUsage(s.ctx, "acme", "", s.base, s.base.Add(24*time.Hour)) + s.Require().NoError(err) + s.Require().Len(tools, 1) + s.InDelta(250, tools[0].AvgTokens, 0.001, "the result it gave the model is gone, not what it cost") + s.Equal(ToolLoadingSummary, tools[0].Loading) + s.Require().NotNil(tools[0].DefinitionTokens) + s.EqualValues(461, *tools[0].DefinitionTokens) + s.Require().NotNil(tools[0].SummaryTokens) + s.EqualValues(84, *tools[0].SummaryTokens) +} + func (s *StoreSuite) TestUsageQueriesRejectBadArguments() { _, err := s.store.CustomerSpend(s.ctx, "", "modality", Daily, s.base, s.base.Add(time.Hour), 6, nil, UsageFilter{}) s.ErrorContains(err, "customer id is required") diff --git a/acceleration/migrations/20270102090000_agent_config_tool_groups.sql b/acceleration/migrations/20270102090000_agent_config_tool_groups.sql new file mode 100644 index 0000000000..40d03143eb --- /dev/null +++ b/acceleration/migrations/20270102090000_agent_config_tool_groups.sql @@ -0,0 +1,44 @@ +-- +goose NO TRANSACTION +-- +goose Up + +-- tool_groups are the functions an SDK provides that a config's sessions may offer, each +-- group with how it is loaded and how long what its tools were given and gave back is kept +-- (store.AgentConfig.ToolGroups). Null, as on every config made before them, names none, so +-- such a config's sessions offer no function until it names one. No default, so adding the +-- column rewrites no row; the lock waits at most 5 s behind a running query, past which the +-- migration fails and the router's next start tries again. +-- +goose StatementBegin +DO $$ +BEGIN + SET LOCAL lock_timeout = '5s'; + ALTER TABLE agent_configs ADD COLUMN IF NOT EXISTS tool_groups JSONB; +END $$; +-- +goose StatementEnd + +-- What progressive_tools and tool_output_retention said of a config's connector tools moves +-- onto each of its connector bindings, as loading summary and output_retention. A binding +-- that already says either keeps what it says. Both columns stay, unread from here on, for +-- a router still running the release before. +UPDATE agent_configs AS config +SET connectors = ( + SELECT jsonb_agg( + binding + || CASE WHEN config.progressive_tools AND NOT binding ? 'loading' + THEN '{"loading": "summary"}'::jsonb ELSE '{}'::jsonb END + || CASE WHEN config.tool_output_retention <> '' AND NOT binding ? 'output_retention' + THEN jsonb_build_object('output_retention', config.tool_output_retention) ELSE '{}'::jsonb END + ORDER BY position) + FROM jsonb_array_elements(config.connectors) WITH ORDINALITY AS bound(binding, position) +) +WHERE (config.progressive_tools OR config.tool_output_retention <> '') + AND jsonb_typeof(config.connectors) = 'array' + AND jsonb_array_length(config.connectors) > 0; + +-- +goose Down +-- +goose StatementBegin +DO $$ +BEGIN + SET LOCAL lock_timeout = '5s'; + ALTER TABLE agent_configs DROP COLUMN IF EXISTS tool_groups; +END $$; +-- +goose StatementEnd diff --git a/examples/text_agents/mcp_plugins/agent.yaml b/examples/text_agents/mcp_plugins/agent.yaml index 9b187f5df8..d373dcc182 100644 --- a/examples/text_agents/mcp_plugins/agent.yaml +++ b/examples/text_agents/mcp_plugins/agent.yaml @@ -3,11 +3,9 @@ description: Reads the team's issues and your own calendar and files, and makes mode: text llm: llm-fast # This agent has dozens of tools, and some servers write a page of instructions and examples -# into a tool's description. progressive offers each tool by the first line of its -# description instead, and the first call to a tool returns the rest without running it, -# which costs one more model turn per tool used. Off when left out. -tools: - progressive: true +# into a tool's description. loading: summary offers each of a connector's tools by the first +# line of its description instead, and the first call to a tool returns the rest without +# running it, which costs one more model turn per tool used. full when left out. # Sentry, GitHub, HubSpot and Salesforce are the company's accounts, connected once on # the dashboard. connection: company is that connection's label, resolved when the folder # syncs, so every conversation uses it. A name alone is a session binding: the agent asks @@ -18,22 +16,33 @@ tools: # configured on the router: ask about an issue and the login works. connectors: - name: sentry + loading: summary connection: company tools: [find_organizations, find_projects, search_issues, search_errors, get_sentry_resource, analyze_issue_with_seer, update_issue] - name: github + loading: summary connection: company - name: hubspot + loading: summary connection: company - name: salesforce + loading: summary connection: company - name: linear + loading: summary tools: [list_issues, get_issue, save_issue, list_comments, save_comment, list_projects, get_project, list_cycles, list_teams, list_users, list_issue_statuses] - - google_calendar - - google_drive - - google_docs - - calendly - - calcom - - slack + - name: google_calendar + loading: summary + - name: google_drive + loading: summary + - name: google_docs + loading: summary + - name: calendly + loading: summary + - name: calcom + loading: summary + - name: slack + loading: summary # Blender for the render skill, in the router's Daytona sandbox. sandbox: daytona sandbox_options: diff --git a/examples/text_agents/travel_support/README.md b/examples/text_agents/travel_support/README.md new file mode 100644 index 0000000000..23538692df --- /dev/null +++ b/examples/text_agents/travel_support/README.md @@ -0,0 +1,41 @@ +# Travel support chat agent + +A customer support chatbot for a travel agency, set up from this directory and tested in the +dashboard's chat. [`../travel_support_js`](../travel_support_js) is the same agent in Node. + +| File | What it is | +| ---------------------------- | -------------------------------------------------------------- | +| `agent.yaml` | A text agent with the TableJourney connector and its tools | +| `instructions.md` | Who the agent is and how it answers | +| `knowledge/travel_policy.md` | Bookings, changes, refunds, baggage, insurance and contact | +| `skills/itinerary.md` | How it plans a day-by-day trip, and the markdown it answers in | + +[TableJourney](https://tablejourney.com/agents/) is a public MCP server with verified +restaurants, sights, tours and stays. It is not in the router's catalog, so the script creates +it as a custom connector, `custom_tablejourney`, with an app connection labelled +`tablejourney`. Both are created once; later runs leave them as they are. `agent.yaml` offers +only its read-only tools. + +`lookup_booking` in `travel_support.py` is a function the agent calls, run by this process. +After syncing, the script keeps running as a worker that hosts it (`client.run_dispatch()`), +so the router offers it to every conversation whose agent.yaml names it, the ones the dashboard +opens included, and sends each call here. + +## Run it + +You need a router (see [acceleration/README.md](../../../acceleration/README.md)) and the +repo-root `.env` with `STREAM_ACCELERATION_URL`, `STREAM_API_KEY`, `STREAM_API_SECRET` and +`DASHBOARD_BASE_URL`, the app's agents page on the dashboard. A local router also needs +`ROUTER_CONNECTORS_ENABLED=true`, or it refuses the connector. + +```bash +uv sync +uv run travel_support.py +``` + +The script syncs the agent, opens its test page on the dashboard and hosts `lookup_booking` +until you stop it. Try: + +- "How much does it cost to change my flight?" +- "Which flight am I on? My booking is WF7K2Q." +- "Plan 2 days in Lisbon for a food lover." diff --git a/examples/text_agents/travel_support/__init__.py b/examples/text_agents/travel_support/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/examples/text_agents/travel_support/agent.yaml b/examples/text_agents/travel_support/agent.yaml new file mode 100644 index 0000000000..d9f4a090f4 --- /dev/null +++ b/examples/text_agents/travel_support/agent.yaml @@ -0,0 +1,15 @@ +name: travel_support +description: Travel customer support that answers policy questions and plans itineraries. +mode: text +llm: llm-fast +connectors: + - name: tablejourney + connector: custom_tablejourney + connection: tablejourney + tools: [cities, plan_day, itineraries, search_places, place, things_to_do, experiences, stays, dishes, festivals] +tools: + - name: booking + loading: full + tools: [lookup_booking] + + \ No newline at end of file diff --git a/examples/text_agents/travel_support/instructions.md b/examples/text_agents/travel_support/instructions.md new file mode 100644 index 0000000000..38ce8f1bae --- /dev/null +++ b/examples/text_agents/travel_support/instructions.md @@ -0,0 +1,13 @@ +You are the customer support assistant of Wayfarer Travel, a travel agency that books flights, +hotels and tours. + +Answer questions about bookings, changes, refunds, baggage, insurance and documents from the +knowledge base. Quote the policy's numbers (fees, deadlines, weights) exactly, and say so when +the policy does not cover a question instead of guessing. When someone asks about their own +trip, ask for their booking reference and look it up with `lookup_booking`. For anything you cannot do in this +chat, such as changing a booking, give the contact details from the policy. + +When someone wants a trip planned, follow the itinerary skill. Use the TableJourney tools for +real places to eat, see and stay; never invent a venue. + +Write in markdown. Keep answers short: a few sentences, or a list when there are steps. diff --git a/examples/text_agents/travel_support/knowledge/travel_policy.md b/examples/text_agents/travel_support/knowledge/travel_policy.md new file mode 100644 index 0000000000..3351757437 --- /dev/null +++ b/examples/text_agents/travel_support/knowledge/travel_policy.md @@ -0,0 +1,50 @@ +# Wayfarer Travel customer policy + +## Bookings + +- A booking is confirmed once paid in full. The confirmation email carries a six-character + booking reference, such as `WF7K2Q`. +- Package holidays can be held for 24 hours without payment. After that the price is no longer + guaranteed. + +## Changes + +- Dates or names on a flight booking can be changed up to 48 hours before departure for a fee of + €40 per traveller, plus any fare difference. +- Hotel-only bookings can be changed free of charge up to 7 days before check-in. +- Changes within 48 hours of departure are handled by phone only. + +## Cancellation and refunds + +| When you cancel | Refund | +| -------------------------------- | -------------------------- | +| More than 30 days before travel | Full refund minus €25 fee | +| 8 to 30 days before travel | 50% of the booking | +| 7 days or fewer before travel | No refund | + +- Refunds go back to the original payment method within 10 working days. +- If Wayfarer or an airline cancels, the refund is always in full, with no fee. + +## Baggage + +- Every flight includes one personal item (40 × 30 × 20 cm). +- Economy fares include one cabin bag of up to 8 kg. A checked bag of up to 23 kg costs €35 per + flight when added online, €60 at the airport. +- Sports equipment and musical instruments must be added at least 72 hours before departure. + +## Travel insurance + +- Optional cover costs 4% of the booking and must be added within 14 days of booking. +- It covers cancellation for illness, missed connections and lost luggage up to €1,500. + +## Documents and visas + +- Travellers are responsible for valid passports and visas. Most destinations require a passport + valid for at least 6 months after the return date. +- Wayfarer can check entry requirements, but cannot apply for visas on a traveller's behalf. + +## Contact + +- Phone: +31 20 555 0142, every day from 08:00 to 22:00 CET. +- Email: support@wayfarer.example, answered within one working day. +- Emergencies while travelling: +31 20 555 0199, 24 hours a day. diff --git a/examples/text_agents/travel_support/pyproject.toml b/examples/text_agents/travel_support/pyproject.toml new file mode 100644 index 0000000000..d7e4dc8f2f --- /dev/null +++ b/examples/text_agents/travel_support/pyproject.toml @@ -0,0 +1,16 @@ +[project] +name = "travel-support-agent" +version = "0.0.0" +requires-python = ">=3.13,<3.14" + +dependencies = [ + "python-dotenv>=1.0", + "vision-agents-plugins-getstream", + "vision-agents-plugins-stream", + "vision-agents", +] + +[tool.uv.sources] +"vision-agents-plugins-getstream" = {path = "../../../plugins/getstream", editable=true} +"vision-agents-plugins-stream" = {path = "../../../plugins/stream", editable=true} +"vision-agents" = {path = "../../../sdks/python", editable=true} diff --git a/examples/text_agents/travel_support/skills/itinerary.md b/examples/text_agents/travel_support/skills/itinerary.md new file mode 100644 index 0000000000..10fe3a3860 --- /dev/null +++ b/examples/text_agents/travel_support/skills/itinerary.md @@ -0,0 +1,30 @@ +--- +description: Plan a day-by-day trip itinerary for a city, with real places to eat, see and stay. +deadline: 2m +--- +You plan itineraries. Before planning, make sure you know the destination city, the dates or +number of days, and what the traveller enjoys. Ask once for whatever is missing; budget and +dietary needs are worth asking about too, but plan without them if they don't say. + +Then use the TableJourney tools: + +1. `cities` when you are not sure the city is covered. If it is not, say so and offer the + nearest covered city. +2. `itineraries` for the city's editorial plans, which are the better base when one fits. +3. `plan_day` for each day, passing the date, dietary need and price tier. +4. `things_to_do` and `experiences` for the hours between meals, and `stays` when they still + need a place to sleep. + +Use only places the tools returned, and link each one to the page URL or booking link it came +with. + +Answer in exactly this format, one section per day: + +## Day 1 - Saturday 12 June + +| Time | Stop | Why | Link | +| ----- | ----------- | ----------------------- | ------------------- | +| 09:00 | Place name | One line on why it fits | [Book](https://...) | + +After the last day, add a short **Book ahead** list of the stops that need a reservation, then +one sentence asking whether they want anything changed. diff --git a/examples/text_agents/travel_support/travel_support.py b/examples/text_agents/travel_support/travel_support.py new file mode 100644 index 0000000000..899cd3c924 --- /dev/null +++ b/examples/text_agents/travel_support/travel_support.py @@ -0,0 +1,106 @@ +import asyncio +import logging +import os +import webbrowser +from pathlib import Path + +from dotenv import load_dotenv +from vision_agents.plugins import stream + +logging.basicConfig(level=logging.INFO) +logging.getLogger("httpx").setLevel(logging.WARNING) + +load_dotenv() + +""" +A travel support chat agent, set up from this directory and tested on the dashboard. + +`instructions.md` is who it is, `knowledge/travel_policy.md` what it knows about bookings, +refunds and baggage, and `skills/itinerary.md` how it plans a trip. TableJourney, a public +MCP server with real restaurants, sights and stays, is a custom connector the app creates +once; `agent.yaml` binds it and names the tools the agent may call. + +`lookup_booking` is a function of this process: it stays running and hosts it, so every +conversation with the agent can call it, the ones the dashboard opens included. + +This script makes sure the connector exists, syncs the agent and opens the dashboard's test +page, where you chat with it. Needs a router (STREAM_ACCELERATION_URL), Stream credentials +and DASHBOARD_BASE_URL, the app's agents page on the dashboard. +""" + +HERE = Path(__file__).parent +BOOKINGS = { + "WF7K2Q": { + "traveller": "Ana Silva", + "trip": "Amsterdam to Lisbon, 12 to 15 June", + "flight": "TP 667, departs 07:05", + "hotel": "Casa do Bairro, 3 nights", + "checked_bags": 1, + "insurance": False, + "status": "confirmed", + }, +} +CONNECTOR = { + "id": "custom_tablejourney", + "name": "TableJourney", + "description": "Verified restaurants, food festivals and food tours", + "endpoint": "https://tablejourney.com/mcp", + "schemes": ["none"], +} +CONNECTION_LABEL = "tablejourney" + +client = stream.Client() + + +@client.tools.register() +async def lookup_booking(reference: str) -> dict[str, object]: + """Look up a booking by its six-character booking reference, such as WF7K2Q.""" + booking = BOOKINGS.get(reference.strip().upper()) + if booking is None: + return {"error": f"no booking with reference {reference}"} + return {"reference": reference.strip().upper(), **booking} + + +async def main() -> None: + await ensure_tablejourney(stream.Backend()) + synced = await stream.sync_agent("travel_support", path=str(HERE)) + + url = f"{os.environ['DASHBOARD_BASE_URL'].rstrip('/')}/{synced.config.id}/test?channel=text" + print(f"Chat with travel_support: {url}") + webbrowser.open(url, 2) + + print("Hosting lookup_booking; ctrl-c to stop.") + await client.run_dispatch() + + +async def ensure_tablejourney(backend: stream.Backend) -> None: + """Create the TableJourney connector and the app's connection to it, once.""" + async with backend.client().get_async_httpx_client() as http: + # Creating it again with the same definition stores nothing new. + await http.post("/v1/agents/connectors", json=CONNECTOR) + + listed = await http.get( + "/v1/agents/connections", + params={"connector_id": CONNECTOR["id"], "owner_type": "app"}, + ) + if any(c.get("label") == CONNECTION_LABEL for c in listed.json()["items"]): + return + + created = await http.post( + "/v1/agents/connections", + json={ + "connector_id": CONNECTOR["id"], + "owner": {"type": "app"}, + "label": CONNECTION_LABEL, + }, + ) + connection = created.json() + # A connector that takes no login is connected by storing empty credentials. + await http.put( + f"/v1/agents/connections/{connection['id']}/credentials", + json={"expected_revision": connection["revision"]}, + ) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/examples/text_agents/travel_support_js/.gitignore b/examples/text_agents/travel_support_js/.gitignore new file mode 100644 index 0000000000..3c3629e647 --- /dev/null +++ b/examples/text_agents/travel_support_js/.gitignore @@ -0,0 +1 @@ +node_modules diff --git a/examples/text_agents/travel_support_js/README.md b/examples/text_agents/travel_support_js/README.md new file mode 100644 index 0000000000..afe77b33c9 --- /dev/null +++ b/examples/text_agents/travel_support_js/README.md @@ -0,0 +1,40 @@ +# Travel support chat agent (Node) + +A customer support chatbot for a travel agency, set up from this directory and tested in the +dashboard's chat. [`../travel_support`](../travel_support) is the same agent in Python. + +| File | What it is | +| ---------------------------- | -------------------------------------------------------------- | +| `agent.yaml` | A text agent with the TableJourney connector and its tools | +| `instructions.md` | Who the agent is and how it answers | +| `knowledge/travel_policy.md` | Bookings, changes, refunds, baggage, insurance and contact | +| `skills/itinerary.md` | How it plans a day-by-day trip, and the markdown it answers in | + +[TableJourney](https://tablejourney.com/agents/) is a public MCP server with verified +restaurants, sights, tours and stays. It is not in the router's catalog, so the script creates +it as a custom connector, `custom_tablejourney`, with an app connection labelled +`tablejourney`. Both are created once; later runs leave them as they are. `agent.yaml` offers +only its read-only tools. + +`lookup_booking` in `index.mjs` is a function the agent calls, run by this process. After +syncing, the script keeps running as a worker that hosts it (`client.runDispatch()`), so the +router offers it to every conversation whose agent.yaml names it, the ones the dashboard opens +included, and sends each call here. + +## Run it + +You need Node 22, a router (see [acceleration/README.md](../../../acceleration/README.md)) and +the repo-root `.env` with `STREAM_ACCELERATION_URL`, `STREAM_API_KEY`, `STREAM_API_SECRET` and +`DASHBOARD_BASE_URL`, the app's agents page on the dashboard. A local router also needs +`ROUTER_CONNECTORS_ENABLED=true`, or it refuses the connector. + +```bash +npm install +npm start +``` + +Try: + +- "How much does it cost to change my flight?" +- "Which flight am I on? My booking is WF7K2Q." +- "Plan 2 days in Lisbon for a food lover." diff --git a/examples/text_agents/travel_support_js/agent.yaml b/examples/text_agents/travel_support_js/agent.yaml new file mode 100644 index 0000000000..8229c40825 --- /dev/null +++ b/examples/text_agents/travel_support_js/agent.yaml @@ -0,0 +1,9 @@ +name: travel_support +description: Travel customer support that answers policy questions and plans itineraries. +mode: text +llm: llm-fast +connectors: + - name: tablejourney + connector: custom_tablejourney + connection: tablejourney + tools: [cities, plan_day, itineraries, search_places, place, things_to_do, experiences, stays, dishes, festivals] diff --git a/examples/text_agents/travel_support_js/index.mjs b/examples/text_agents/travel_support_js/index.mjs new file mode 100644 index 0000000000..a4677193cf --- /dev/null +++ b/examples/text_agents/travel_support_js/index.mjs @@ -0,0 +1,89 @@ +// A travel support chat agent, set up from this directory and tested on the dashboard. +// +// instructions.md is who it is, knowledge/travel_policy.md what it knows about bookings, refunds +// and baggage, and skills/itinerary.md how it plans a trip. TableJourney, a public MCP server +// with real restaurants, sights and stays, is a custom connector the app creates once; agent.yaml +// binds it and names the tools the agent may call. This script makes sure the connector exists, +// syncs the agent and opens the dashboard's test page, where you chat with it. +// `lookup_booking` is a function of this process: it stays running and hosts it, so every +// conversation with the agent can call it, the ones the dashboard opens included. +// Needs a router (STREAM_ACCELERATION_URL), Stream credentials and DASHBOARD_BASE_URL, the app's +// agents page on the dashboard. + +import { spawn } from "node:child_process"; +import { fileURLToPath } from "node:url"; + +import { Agent } from "@stream-io/stream-agents"; +import { loadFolder } from "@stream-io/stream-agents/node"; + +const FOLDER = fileURLToPath(new URL(".", import.meta.url)); +const CONNECTOR = { + id: "custom_tablejourney", + name: "TableJourney", + description: "Verified restaurants, food festivals and food tours", + endpoint: "https://tablejourney.com/mcp", + schemes: ["none"], +}; +const CONNECTION_LABEL = "tablejourney"; +const BOOKINGS = { + WF7K2Q: { + traveller: "Ana Silva", + trip: "Amsterdam to Lisbon, 12 to 15 June", + flight: "TP 667, departs 07:05", + hotel: "Casa do Bairro, 3 nights", + checked_bags: 1, + insurance: false, + status: "confirmed", + }, +}; + +const agent = new Agent({ folder: await loadFolder(FOLDER) }); +await ensureTablejourney(agent.client); +const synced = await agent.sync(); + +const url = `${process.env.DASHBOARD_BASE_URL.replace(/\/$/, "")}/${synced.config.id}/test?channel=text`; +console.log(`Chat with travel_support: ${url}`); +openInBrowser(url); + +agent.client.tools.register({ + name: "lookup_booking", + description: "Look up a booking by its six-character booking reference, such as WF7K2Q.", + parameters: { type: "object", properties: { reference: { type: "string" } }, required: ["reference"] }, + run: async ({ reference }) => { + const key = reference.trim().toUpperCase(); + const booking = BOOKINGS[key]; + return booking ? { reference: key, ...booking } : { error: `no booking with reference ${reference}` }; + }, +}); + +const stop = new AbortController(); +process.on("SIGINT", () => stop.abort()); +console.log("Hosting lookup_booking; ctrl-c to stop."); +await agent.client.runDispatch(stop.signal); + +/** Creates the TableJourney connector and the app's connection to it, once. */ +async function ensureTablejourney(client) { + // Creating it again with the same definition stores nothing new. + await client.post("/v1/agents/connectors", { body: CONNECTOR }); + + const listed = await client.get("/v1/agents/connections", { + query: { connector_id: CONNECTOR.id, owner_type: "app" }, + }); + if (listed.items.some((connection) => connection.label === CONNECTION_LABEL)) { + return; + } + + const connection = await client.post("/v1/agents/connections", { + body: { connector_id: CONNECTOR.id, owner: { type: "app" }, label: CONNECTION_LABEL }, + }); + // A connector that takes no login is connected by storing empty credentials. + await client.put("/v1/agents/connections/{id}/credentials", { + path: { id: connection.id }, + body: { expected_revision: connection.revision }, + }); +} + +function openInBrowser(url) { + const command = { darwin: "open", win32: "explorer" }[process.platform] ?? "xdg-open"; + spawn(command, [url], { stdio: "ignore", detached: true }).unref(); +} diff --git a/examples/text_agents/travel_support_js/instructions.md b/examples/text_agents/travel_support_js/instructions.md new file mode 100644 index 0000000000..38ce8f1bae --- /dev/null +++ b/examples/text_agents/travel_support_js/instructions.md @@ -0,0 +1,13 @@ +You are the customer support assistant of Wayfarer Travel, a travel agency that books flights, +hotels and tours. + +Answer questions about bookings, changes, refunds, baggage, insurance and documents from the +knowledge base. Quote the policy's numbers (fees, deadlines, weights) exactly, and say so when +the policy does not cover a question instead of guessing. When someone asks about their own +trip, ask for their booking reference and look it up with `lookup_booking`. For anything you cannot do in this +chat, such as changing a booking, give the contact details from the policy. + +When someone wants a trip planned, follow the itinerary skill. Use the TableJourney tools for +real places to eat, see and stay; never invent a venue. + +Write in markdown. Keep answers short: a few sentences, or a list when there are steps. diff --git a/examples/text_agents/travel_support_js/knowledge/travel_policy.md b/examples/text_agents/travel_support_js/knowledge/travel_policy.md new file mode 100644 index 0000000000..3351757437 --- /dev/null +++ b/examples/text_agents/travel_support_js/knowledge/travel_policy.md @@ -0,0 +1,50 @@ +# Wayfarer Travel customer policy + +## Bookings + +- A booking is confirmed once paid in full. The confirmation email carries a six-character + booking reference, such as `WF7K2Q`. +- Package holidays can be held for 24 hours without payment. After that the price is no longer + guaranteed. + +## Changes + +- Dates or names on a flight booking can be changed up to 48 hours before departure for a fee of + €40 per traveller, plus any fare difference. +- Hotel-only bookings can be changed free of charge up to 7 days before check-in. +- Changes within 48 hours of departure are handled by phone only. + +## Cancellation and refunds + +| When you cancel | Refund | +| -------------------------------- | -------------------------- | +| More than 30 days before travel | Full refund minus €25 fee | +| 8 to 30 days before travel | 50% of the booking | +| 7 days or fewer before travel | No refund | + +- Refunds go back to the original payment method within 10 working days. +- If Wayfarer or an airline cancels, the refund is always in full, with no fee. + +## Baggage + +- Every flight includes one personal item (40 × 30 × 20 cm). +- Economy fares include one cabin bag of up to 8 kg. A checked bag of up to 23 kg costs €35 per + flight when added online, €60 at the airport. +- Sports equipment and musical instruments must be added at least 72 hours before departure. + +## Travel insurance + +- Optional cover costs 4% of the booking and must be added within 14 days of booking. +- It covers cancellation for illness, missed connections and lost luggage up to €1,500. + +## Documents and visas + +- Travellers are responsible for valid passports and visas. Most destinations require a passport + valid for at least 6 months after the return date. +- Wayfarer can check entry requirements, but cannot apply for visas on a traveller's behalf. + +## Contact + +- Phone: +31 20 555 0142, every day from 08:00 to 22:00 CET. +- Email: support@wayfarer.example, answered within one working day. +- Emergencies while travelling: +31 20 555 0199, 24 hours a day. diff --git a/examples/text_agents/travel_support_js/package-lock.json b/examples/text_agents/travel_support_js/package-lock.json new file mode 100644 index 0000000000..283e303f49 --- /dev/null +++ b/examples/text_agents/travel_support_js/package-lock.json @@ -0,0 +1,39 @@ +{ + "name": "travel-support-agent", + "version": "0.0.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "travel-support-agent", + "version": "0.0.0", + "dependencies": { + "@stream-io/stream-agents": "^0.4.3" + }, + "engines": { + "node": ">=22" + } + }, + "node_modules/@stream-io/stream-agents": { + "version": "0.4.3", + "resolved": "https://registry.npmjs.org/@stream-io/stream-agents/-/stream-agents-0.4.3.tgz", + "integrity": "sha512-tt6RpqLKc2qBndmPQaLBKn8RsXDgZ7PZwOHtn5ssYNZNvG37raBkP6zzBbHYDjYo/EdRkQAXciY/3ZEgFmE/CQ==", + "license": "Apache-2.0", + "engines": { + "node": ">=22" + }, + "peerDependencies": { + "@stream-io/video-client": "^1.0.0", + "stream-chat": "^9.0.0 || ^10.0.2" + }, + "peerDependenciesMeta": { + "@stream-io/video-client": { + "optional": true + }, + "stream-chat": { + "optional": true + } + } + } + } +} diff --git a/examples/text_agents/travel_support_js/package.json b/examples/text_agents/travel_support_js/package.json new file mode 100644 index 0000000000..d0356dd9ff --- /dev/null +++ b/examples/text_agents/travel_support_js/package.json @@ -0,0 +1,15 @@ +{ + "name": "travel-support-agent", + "version": "0.0.0", + "private": true, + "type": "module", + "engines": { + "node": ">=22" + }, + "scripts": { + "start": "node --env-file=../../../.env index.mjs" + }, + "dependencies": { + "@stream-io/stream-agents": "^0.4.3" + } +} diff --git a/examples/text_agents/travel_support_js/skills/itinerary.md b/examples/text_agents/travel_support_js/skills/itinerary.md new file mode 100644 index 0000000000..10fe3a3860 --- /dev/null +++ b/examples/text_agents/travel_support_js/skills/itinerary.md @@ -0,0 +1,30 @@ +--- +description: Plan a day-by-day trip itinerary for a city, with real places to eat, see and stay. +deadline: 2m +--- +You plan itineraries. Before planning, make sure you know the destination city, the dates or +number of days, and what the traveller enjoys. Ask once for whatever is missing; budget and +dietary needs are worth asking about too, but plan without them if they don't say. + +Then use the TableJourney tools: + +1. `cities` when you are not sure the city is covered. If it is not, say so and offer the + nearest covered city. +2. `itineraries` for the city's editorial plans, which are the better base when one fits. +3. `plan_day` for each day, passing the date, dietary need and price tier. +4. `things_to_do` and `experiences` for the hours between meals, and `stays` when they still + need a place to sleep. + +Use only places the tools returned, and link each one to the page URL or booking link it came +with. + +Answer in exactly this format, one section per day: + +## Day 1 - Saturday 12 June + +| Time | Stop | Why | Link | +| ----- | ----------- | ----------------------- | ------------------- | +| 09:00 | Place name | One line on why it fits | [Book](https://...) | + +After the last day, add a short **Book ahead** list of the stops that need a reservation, then +one sentence asking whether they want anything changed. diff --git a/examples/voice_agents/crew_assistant/agent.yaml b/examples/voice_agents/crew_assistant/agent.yaml index 6ab55f5ad4..6f4560c820 100644 --- a/examples/voice_agents/crew_assistant/agent.yaml +++ b/examples/voice_agents/crew_assistant/agent.yaml @@ -6,6 +6,14 @@ tts: inworld/inworld-tts-2-flash greeting: text: Hey, it's Crew. What do you need before, during or after your shift? mode: exact +# The restaurant's tools, hosted by the backend, and the phone's own (get_local_context and the +# three that ask the person first). +tools: + - name: crew + loading: full + tools: [get_local_context, get_my_schedule, check_callout, find_coverage, notify_shift_leader, + search_handbook, get_updates_since, get_equipment_guide, get_team, get_menu, + save_shift_reflection, request_shift_swap, create_support_ticket, book_check_in] # The person's own Cal.com, connected in the chat the first time the agent needs it, the way # swift_assistant connects Linear: Cal.com registers the router as its client by itself, so the # router needs no OAuth client. The one tool that writes, create_booking, runs only after the diff --git a/examples/voice_agents/crew_assistant/backend/hosted.go b/examples/voice_agents/crew_assistant/backend/hosted.go index 3fefbba7b0..c8f30c33c9 100644 --- a/examples/voice_agents/crew_assistant/backend/hosted.go +++ b/examples/voice_agents/crew_assistant/backend/hosted.go @@ -8,15 +8,13 @@ import ( "github.com/GetStream/Vision-Agents/sdks/go/tools" ) -// hostedAgent is the agent the restaurant's tools are hosted under: the router offers them to -// every session of it and sends their calls here. -type hostedAgent struct { - name string +// restaurantTools are the tools this backend hosts for every agent: the router offers them to +// the sessions of crew_assistant, whose tool group names them, and sends their calls here. +type restaurantTools struct { tools *tools.Registry } -func (a hostedAgent) Name() string { return a.name } -func (a hostedAgent) Tools() *tools.Registry { return a.tools } +func (r restaurantTools) Tools() *tools.Registry { return r.tools } type ( none struct{} diff --git a/examples/voice_agents/crew_assistant/backend/main.go b/examples/voice_agents/crew_assistant/backend/main.go index 8c9abced14..76ae3942bc 100644 --- a/examples/voice_agents/crew_assistant/backend/main.go +++ b/examples/voice_agents/crew_assistant/backend/main.go @@ -82,7 +82,7 @@ func run(ctx context.Context, addr, simulationUser string) error { if err != nil { return err } - dispatch.Host(hostedAgent{name: agentName, tools: hosted}, 30*time.Second) + dispatch.Host(restaurantTools{tools: hosted}, 30*time.Second) mux := http.NewServeMux() s.routes(mux) diff --git a/plugins/stream/README.md b/plugins/stream/README.md index b71a580e44..309cb7ca4c 100644 --- a/plugins/stream/README.md +++ b/plugins/stream/README.md @@ -102,7 +102,6 @@ from vision_agents.core.llm import ToolCallContext needs_approval="Book flight {flight_id}?", ) async def book_flight(tool_call_context: ToolCallContext, flight_id: str) -> str: - await tool_call_context.update("checking seats") await tool_call_context.disallow_interruptions() return await bookings.book(flight_id, user=tool_call_context.user.id, key=tool_call_context.id) ``` @@ -426,31 +425,30 @@ team. A token's team always wins, then the session's, then the agent's tag. - **Refusals** carry `code`: `limit_above_parent`, `limit_exists`, `no_limit`, `use_team`, `invalid_expiry`, `too_many_grants` and `server_side_only`. -## Tools hosted for an agent +## Tools -A session's own functions run in the process that opened it. `dispatch.host` is the other -direction: the router offers an agent's functions to every session under that agent, -wherever it was opened, and sends each call to this worker. +Tools are registered once for the app, on the client. Each agent is offered the ones a tool +group of its `agent.yaml` names, in every session, wherever it was opened, and each call is +sent to a worker running `run_dispatch`. ```python -agent = stream.Client().agent("stream-support") +client = stream.Client() -@agent.register(description="Read the SDK's source") +@client.tools.register(description="Read the SDK's source", timeout=60) async def investigate_sdk(sdk: str) -> str: return await read_source(sdk) -dispatch = stream.Dispatch() -dispatch.host(agent, tool_timeout=60) -await dispatch.run() +await client.run_dispatch() ``` -`tool_timeout` is how long the router gives one call, in seconds; zero takes its default -of 30. A hosted function takes the same `interruptible`, `timeout` and `needs_approval` as +A tool's `timeout` is how long the router gives one call, in seconds; zero takes its default +of 30. A tool takes the same `interruptible`, `timeout` and `needs_approval` as `register_function`, can take its call as its first argument, and is cancelled when the router -gives up on it. The worker declares its tools again whenever it reconnects, and `run` raises `RouterError` if the -router refuses them. +gives up on it. The worker declares its tools again whenever it reconnects, and `run_dispatch` +raises `RouterError` if the router refuses them. A worker that also answers calls or messages +hosts them with `dispatch.host(client)` before `dispatch.run()`. ## Calling somebody diff --git a/plugins/stream/tests/test_accelerated.py b/plugins/stream/tests/test_accelerated.py index 78ce791b4a..6ebdccceb9 100644 --- a/plugins/stream/tests/test_accelerated.py +++ b/plugins/stream/tests/test_accelerated.py @@ -920,33 +920,6 @@ async def whoami(tool_call_context: ToolCallContext) -> str: ) assert seen[0].approved - async def test_a_function_tells_the_caller_how_it_is_going( - self, router: Router, joined: stream.Accelerated - ): - @joined.register_function(description="Search flights") - async def search(tool_call_context: ToolCallContext) -> str: - await tool_call_context.update("checking three airlines") - await tool_call_context.say("Still looking, one moment.") - return "found two" - - await router.send( - {"type": "tool_call", "id": "call-4", "name": "search", "arguments": "{}"} - ) - - assert await router.answered() == { - "type": "tool_update", - "tool_call_id": "call-4", - "text": "checking three airlines", - "verbatim": False, - } - assert await router.answered() == { - "type": "tool_update", - "tool_call_id": "call-4", - "text": "Still looking, one moment.", - "verbatim": True, - } - assert (await router.answered())["output"] == "found two" - async def test_a_held_call_is_not_cancelled_by_the_caller_talking_over( self, router: Router, joined: stream.Accelerated ): diff --git a/plugins/stream/tests/test_client.py b/plugins/stream/tests/test_client.py index 3933b1589f..cf2b93bece 100644 --- a/plugins/stream/tests/test_client.py +++ b/plugins/stream/tests/test_client.py @@ -466,19 +466,26 @@ def api(router: Router) -> stream.Client: @pytest.fixture def docs(api: stream.Client) -> stream.Agent: - """An agent whose own lookup_order every one of its sessions runs.""" - agent = api.agent("docs") + return api.agent("docs") - @agent.register(description="Look up an order") + +@pytest.fixture +def lookup() -> FunctionRegistry: + """The caller's own lookup_order, which the session below runs.""" + functions = FunctionRegistry() + + @functions.register(description="Look up an order") async def lookup_order() -> str: return "found" - return agent + return functions @pytest.fixture -async def session(docs: stream.Agent) -> AsyncIterator[stream.Session]: - session = await docs.sessions.create(stream.SessionOptions()) +async def session( + docs: stream.Agent, lookup: FunctionRegistry +) -> AsyncIterator[stream.Session]: + session = await docs.sessions.create(stream.SessionOptions(tools=lookup)) yield session await session.close() @@ -548,19 +555,13 @@ async def test_a_session_is_opened_against_the_agent_by_name( assert field not in body async def test_a_call_for_a_tool_held_elsewhere_is_left_for_whoever_holds_it( - self, api: stream.Client, router: Router + self, docs: stream.Agent, router: Router, lookup: FunctionRegistry ): - agent = api.agent("docs") - - @agent.register(description="Look up an order") - async def lookup_order() -> str: - return "found" - router.frames = [ {"type": "tool_call", "id": "call-1", "name": "book_flight"}, {"type": "tool_call", "id": "call-2", "name": "lookup_order"}, ] - session = await agent.sessions.create(stream.SessionOptions()) + session = await docs.sessions.create(stream.SessionOptions(tools=lookup)) try: answer = await asyncio.wait_for(router.commands.get(), 2) assert answer["tool_call_id"] == "call-2" @@ -574,11 +575,11 @@ async def test_a_call_addressed_here_runs_with_its_call_and_its_policies( ): agent = api.agent("docs") seen: list[stream.ToolCallContext] = [] + functions = FunctionRegistry() - @agent.register(description="Look up an order", timeout=2.5) + @functions.register(description="Look up an order", timeout=2.5) async def lookup_order(tool_call_context: stream.ToolCallContext) -> str: seen.append(tool_call_context) - await tool_call_context.update("searching the archive") return "found" router.frames = [ @@ -591,9 +592,8 @@ async def lookup_order(tool_call_context: stream.ToolCallContext) -> str: "run": True, }, ] - session = await agent.sessions.create(stream.SessionOptions()) + session = await agent.sessions.create(stream.SessionOptions(tools=functions)) try: - update = await asyncio.wait_for(router.commands.get(), 2) answer = await asyncio.wait_for(router.commands.get(), 2) await asyncio.sleep(0.1) assert router.commands.empty(), "the call addressed elsewhere is left" @@ -603,12 +603,6 @@ async def lookup_order(tool_call_context: stream.ToolCallContext) -> str: assert ( router.body("POST", "/v1/agents/sessions")["tools"][0]["timeout_ms"] == 2500 ) - assert update == { - "type": "tool_update", - "tool_call_id": "call-2", - "text": "searching the archive", - "verbatim": False, - } assert answer["tool_call_id"] == "call-2" assert (seen[0].id, seen[0].session_id, seen[0].turn_id) == ( "call-2", @@ -1376,15 +1370,19 @@ async def test_an_add_under_a_name_the_session_runs_keeps_its_function( assert await router.ran("reserve_table") == "Reserved T4." - async def test_the_agents_own_tool_withdrawn_runs_nothing_until_another_takes_its_name( - self, docs: stream.Agent, router: Router, window_table: FunctionRegistry + async def test_a_tool_opened_with_withdrawn_runs_nothing_until_another_takes_its_name( + self, + docs: stream.Agent, + router: Router, + window_table: FunctionRegistry, + lookup: FunctionRegistry, ): - @docs.register(name="reserve_table", description="Reserve a table") + @lookup.register(name="reserve_table", description="Reserve a table") async def reserve(tool_call_context: stream.ToolCallContext, table: str) -> str: router.started.append(tool_call_context.id) return f"Reserved {table}." - session = await docs.sessions.create(stream.SessionOptions()) + session = await docs.sessions.create(stream.SessionOptions(tools=lookup)) try: await session.remove_tool("reserve_table") withdrawn = await router.ran("reserve_table") @@ -1408,8 +1406,7 @@ async def test_a_tool_given_to_one_session_is_not_its_agents( another = await docs.sessions.create(stream.SessionOptions()) await another.close() - declared = router.body("POST", "/v1/agents/sessions")["tools"] - assert [tool["name"] for tool in declared] == ["lookup_order"] + assert "tools" not in router.body("POST", "/v1/agents/sessions") async def test_one_change_of_a_sessions_tools_is_made_at_a_time( self, @@ -1863,14 +1860,16 @@ async def test_forking_at_a_response_branches_from_there( async def test_a_fork_inherits_the_parents_functions( self, api: stream.Client, router: Router ): - agent = api.agent("docs") + functions = FunctionRegistry() - @agent.register() + @functions.register() async def get_weather(location: str) -> str: """Get current weather for a location""" return "raining in " + location - session = await agent.sessions.create() + session = await api.agent("docs").sessions.create( + stream.SessionOptions(tools=functions) + ) try: forked = await session.fork() await forked.close() @@ -1880,16 +1879,20 @@ async def get_weather(location: str) -> str: assert forked.functions.list_functions() == ["get_weather"] async def test_a_fork_runs_the_tools_its_parent_has_now( - self, docs: stream.Agent, router: Router, reserve_table: FunctionRegistry + self, + docs: stream.Agent, + router: Router, + reserve_table: FunctionRegistry, + lookup: FunctionRegistry, ): - @docs.register(description="Release a table") + @lookup.register(description="Release a table") async def release_table( tool_call_context: stream.ToolCallContext, table: str ) -> str: router.started.append(tool_call_context.id) return f"Released {table}." - session = await docs.sessions.create(stream.SessionOptions()) + session = await docs.sessions.create(stream.SessionOptions(tools=lookup)) try: await session.add_tools(reserve_table) await session.remove_tool("release_table") @@ -1932,16 +1935,16 @@ async def test_a_fork_withdraws_a_name_whose_adding_got_no_answer_before_adding_ assert ran == "Reserved T4 by the window." async def test_a_fork_waits_for_a_change_of_its_parents_tools( - self, docs: stream.Agent, router: Router + self, docs: stream.Agent, router: Router, lookup: FunctionRegistry ): - @docs.register(description="Release a table") + @lookup.register(description="Release a table") async def release_table( tool_call_context: stream.ToolCallContext, table: str ) -> str: router.started.append(tool_call_context.id) return f"Released {table}." - session = await docs.sessions.create(stream.SessionOptions()) + session = await docs.sessions.create(stream.SessionOptions(tools=lookup)) try: router.tools_take = 0.2 _, forked = await asyncio.gather( @@ -1956,17 +1959,25 @@ async def release_table( assert released is None - async def test_the_model_is_offered_the_agents_functions( + async def test_a_session_declares_its_own_functions_and_not_the_apps( self, api: stream.Client, router: Router ): - agent = api.agent("docs") + # The app's tools are for every agent: declared on a session, the ones its agent's + # tool groups do not name would have the router refuse it. + @api.tools.register(description="Look up an order") + async def lookup_order() -> str: + return "found" - @agent.register() + functions = FunctionRegistry() + + @functions.register() async def get_weather(location: str) -> str: """Get current weather for a location""" return "raining in " + location - session = await agent.sessions.create() + session = await api.agent("docs").sessions.create( + stream.SessionOptions(tools=functions) + ) await session.close() [declared] = router.body("POST", "/v1/agents/sessions")["tools"] diff --git a/plugins/stream/tests/test_config.py b/plugins/stream/tests/test_config.py index c29abc6756..3b0a551e46 100644 --- a/plugins/stream/tests/test_config.py +++ b/plugins/stream/tests/test_config.py @@ -748,18 +748,19 @@ async def test_a_directory_naming_mcp_servers_is_refused_before_anything_is_sent assert router.bodies == [] - async def test_offering_tools_progressively_is_sent( - self, router: Router, support_dir - ): + async def test_tool_groups_are_sent(self, router: Router, support_dir): (support_dir / "agent.yaml").write_text( - "name: support\ntools:\n progressive: true\n" + "name: support\ntools:\n" + " - name: booking\n loading: summary\n tools: [lookup_booking]\n" ) result = await stream.sync_agent( "support", path=str(support_dir), url=router.url, customer_id="acme" ) - assert router.configs[result.config.id]["tools"] == {"progressive": True} + assert router.configs[result.config.id]["tools"] == [ + {"name": "booking", "loading": "summary", "tools": ["lookup_booking"]} + ] async def test_a_file_saying_nothing_about_tools_sends_nothing( self, router: Router, support_dir diff --git a/plugins/stream/tests/test_dispatch.py b/plugins/stream/tests/test_dispatch.py index 5c5c95e9c0..501e7a3d26 100644 --- a/plugins/stream/tests/test_dispatch.py +++ b/plugins/stream/tests/test_dispatch.py @@ -1110,22 +1110,20 @@ async def answering() -> None: assert finished.is_set() @pytest.fixture - def support(self, router: Router) -> stream.Agent: - agent = stream.Client(url=router.url, customer_id="acme").agent( - "stream-support" - ) + def support(self, router: Router) -> stream.Client: + client = stream.Client(url=router.url, customer_id="acme") - @agent.register(description="Read SDK source") + @client.tools.register(description="Read SDK source") async def investigate_sdk(sdk: str) -> str: if sdk == "broken": raise RuntimeError("the checkout is missing") return f"read {sdk}" - return agent + return client @pytest.fixture async def hosting( - self, router: Router, support: stream.Agent + self, router: Router, support: stream.Client ) -> AsyncIterator[stream.Dispatch]: """A worker that only hosts tools, connected and torn down afterwards.""" worker = stream.Dispatch(url=router.url, customer_id="acme", report_every=0.05) @@ -1140,12 +1138,43 @@ async def test_a_hosted_function_is_declared_to_the_router( ): declared = await router.told_of_type("host_tools") - assert declared["agent_id"] == "stream-support" + assert declared["every_agent"] is True + assert "agent_id" not in declared assert declared["timeout_ms"] == 60000 assert [tool["name"] for tool in declared["tools"]] == ["investigate_sdk"] assert declared["tools"][0]["description"] == "Read SDK source" assert "sdk" in declared["tools"][0]["parameters"]["properties"] + async def test_run_dispatch_runs_the_clients_tools_for_every_agent( + self, router: Router + ): + client = stream.Client(url=router.url, customer_id="acme") + + @client.tools.register(description="Look up the weather") + async def weather_lookup(location: str) -> str: + return f"sunny in {location}" + + running = asyncio.create_task(client.run_dispatch()) + try: + declared = await router.told_of_type("host_tools") + assert declared["every_agent"] is True + assert [tool["name"] for tool in declared["tools"]] == ["weather_lookup"] + + await router.hand_over( + { + "type": "tool_call", + "id": "call-1", + "session_id": "s", + "name": "weather_lookup", + "arguments": '{"location": "Amsterdam"}', + } + ) + answered = await router.told_of_type("tool_result") + assert answered["output"] == "sunny in Amsterdam" + finally: + running.cancel() + await asyncio.gather(running, return_exceptions=True) + async def test_a_hosted_function_is_answered_over_the_dispatch_socket( self, router: Router, hosting: stream.Dispatch ): @@ -1205,9 +1234,9 @@ async def test_a_tool_this_worker_does_not_host_is_answered_with_an_error( async def test_a_hosted_functions_policies_are_declared_to_the_router( self, router: Router ): - agent = stream.Client(url=router.url, customer_id="acme").agent("travel") + client = stream.Client(url=router.url, customer_id="acme") - @agent.register( + @client.tools.register( description="Book a flight", interruptible=True, timeout=90, @@ -1217,7 +1246,7 @@ async def book_flight(flight_id: str) -> str: return "booked" worker = stream.Dispatch(url=router.url, customer_id="acme") - worker.host(agent) + worker.host(client) running = asyncio.create_task(worker.run()) try: declared = await router.told_of_type("host_tools") @@ -1231,11 +1260,11 @@ async def book_flight(flight_id: str) -> str: assert tool["approval"] == {"title": "Book flight {flight_id}?"} async def test_a_hosted_call_the_router_cancels_stops(self, router: Router): - agent = stream.Client(url=router.url, customer_id="acme").agent("support") + client = stream.Client(url=router.url, customer_id="acme") started = asyncio.Event() stopped = asyncio.Event() - @agent.register(description="Investigate for a long time") + @client.tools.register(description="Investigate for a long time") async def investigate() -> str: started.set() try: @@ -1245,7 +1274,7 @@ async def investigate() -> str: stopped.set() worker = stream.Dispatch(url=router.url, customer_id="acme") - worker.host(agent) + worker.host(client) running = asyncio.create_task(worker.run()) try: call = {"type": "tool_call", "id": "call-1", "name": "investigate"} @@ -1269,20 +1298,17 @@ async def investigate() -> str: frame for frame in router.received if frame["type"] == "tool_result" ] - async def test_a_hosted_function_reads_its_call_and_says_how_it_is_going( - self, router: Router - ): - agent = stream.Client(url=router.url, customer_id="acme").agent("support") + async def test_a_hosted_function_reads_its_call(self, router: Router): + client = stream.Client(url=router.url, customer_id="acme") seen: list[stream.ToolCallContext] = [] - @agent.register(description="Investigate") + @client.tools.register(description="Investigate") async def investigate(tool_call_context: stream.ToolCallContext) -> str: seen.append(tool_call_context) - await tool_call_context.update("reading the logs") return "found it" worker = stream.Dispatch(url=router.url, customer_id="acme") - worker.host(agent) + worker.host(client) running = asyncio.create_task(worker.run()) try: await router.hand_over( @@ -1295,18 +1321,11 @@ async def investigate(tool_call_context: stream.ToolCallContext) -> str: "user": {"id": "sam"}, } ) - update = await router.told_of_type("tool_update") result = await router.told_of_type("tool_result") finally: running.cancel() await asyncio.gather(running, return_exceptions=True) - assert update == { - "type": "tool_update", - "tool_call_id": "call-1", - "text": "reading the logs", - "verbatim": False, - } assert result["output"] == "found it" assert (seen[0].id, seen[0].session_id, seen[0].turn_id) == ( "call-1", @@ -1319,23 +1338,21 @@ async def test_a_running_hosted_tool_does_not_hold_up_the_next( self, router: Router ): # The socket a call arrived on is also what delivers the next one. - agent = stream.Client(url=router.url, customer_id="acme").agent( - "stream-support" - ) + client = stream.Client(url=router.url, customer_id="acme") release = asyncio.Event() - @agent.register(description="Wait until released") + @client.tools.register(description="Wait until released") async def slow() -> str: await release.wait() return "slow" - @agent.register(description="Release the slow one") + @client.tools.register(description="Release the slow one") async def fast() -> str: release.set() return "fast" worker = stream.Dispatch(url=router.url, customer_id="acme", report_every=0.05) - worker.host(agent) + worker.host(client) running = asyncio.create_task(worker.run()) try: await router.hand_over({"type": "tool_call", "id": "1", "name": "slow"}) @@ -1358,7 +1375,7 @@ async def busy() -> dict[str, Any]: assert [first["output"], second["output"]] == ["fast", "slow"] async def test_a_worker_the_router_drops_reconnects_and_hosts_again( - self, router: Router, support: stream.Agent + self, router: Router, support: stream.Client ): # A worker that stopped at either drop would leave every session naming the agent # without its tools. @@ -1375,11 +1392,11 @@ async def test_a_worker_the_router_drops_reconnects_and_hosts_again( await asyncio.gather(running, return_exceptions=True) assert router.connections == 3 - assert declared["agent_id"] == "stream-support" + assert declared["every_agent"] is True assert declared["timeout_ms"] == 0 async def test_a_worker_whose_tools_are_refused_stops_waiting( - self, router: Router, support: stream.Agent + self, router: Router, support: stream.Client ): # A worker nobody will call should say so rather than sit connected looking healthy. worker = stream.Dispatch(url=router.url, customer_id="acme") @@ -1389,10 +1406,10 @@ async def test_a_worker_whose_tools_are_refused_stops_waiting( await router.hand_over( { "type": "hosting_refused", - "agent_id": "stream-support", + "every_agent": True, "reason": "hosting no tools is not hosting", } ) - with pytest.raises(stream.RouterError, match="stream-support.*not hosting"): + with pytest.raises(stream.RouterError, match="not hosting"): await asyncio.wait_for(running, SETTLE) diff --git a/plugins/stream/tests/test_folder.py b/plugins/stream/tests/test_folder.py index d1873d5624..2b1d7bcc7a 100644 --- a/plugins/stream/tests/test_folder.py +++ b/plugins/stream/tests/test_folder.py @@ -7,7 +7,7 @@ ConnectorSettings, FloorControlSettings, GreetingSettings, - ToolSettings, + ToolGroupSettings, SandboxSettings, find, load, @@ -658,18 +658,44 @@ def test_the_mcp_plugins_example_loads_its_connectors(self): "list_issue_statuses", ] - def test_progressive_tools_are_read(self, tmp_path: Path): + def test_tool_groups_are_read(self, tmp_path: Path): + root = tmp_path / "triage" + write( + root, + "agent.yaml", + "name: triage\ntools:\n" + " - name: booking\n tools: [lookup_booking]\n" + " - name: refunds\n loading: deferred\n output_retention: 30d\n" + " tools: [refund_booking]\n", + ) + + assert load(root).settings.tools == [ + ToolGroupSettings(name="booking", tools=["lookup_booking"]), + ToolGroupSettings( + name="refunds", + tools=["refund_booking"], + loading="deferred", + output_retention="30d", + ), + ] + + def test_the_tool_settings_of_before_groups_say_where_they_went( + self, tmp_path: Path + ): root = tmp_path / "triage" write(root, "agent.yaml", "name: triage\ntools:\n progressive: true\n") - assert load(root).settings.tools == ToolSettings(progressive=True) + with pytest.raises(ValueError, match="loading: summary"): + load(root) @pytest.mark.parametrize( "declared", [ "tools: true\n", - "tools:\n progressive: sometimes\n", - "tools:\n lazy: true\n", + "tools:\n - name: booking\n", + "tools:\n - tools: [lookup_booking]\n", + "tools:\n - name: booking\n loading: lazy\n tools: [a]\n", + "tools:\n - name: booking\n progressive: true\n tools: [a]\n", ], ) def test_tools_that_cannot_be_read_are_refused(self, tmp_path: Path, declared: str): diff --git a/plugins/stream/tests/test_sts.py b/plugins/stream/tests/test_sts.py index 404b40e694..cc553fd796 100644 --- a/plugins/stream/tests/test_sts.py +++ b/plugins/stream/tests/test_sts.py @@ -88,38 +88,6 @@ async def whoami(tool_call_context: ToolCallContext) -> str: assert (await conversation.told())["output"] == "ok" assert seen[0].id == "call-1" - async def test_a_function_tells_the_caller_how_it_is_going( - self, conversation: Conversation, model: stream.STS - ): - @model.register_function(description="Check the book") - async def check(tool_call_context: ToolCallContext) -> str: - await tool_call_context.update("checking the patio") - await tool_call_context.say("One moment.") - return "a table at 7:30" - - await model.connect() - await conversation.send( - {"type": "tool_call", "id": "call-2", "name": "check", "arguments": "{}"} - ) - - assert await conversation.told() == { - "type": "tool_update", - "tool_call_id": "call-2", - "text": "checking the patio", - "verbatim": False, - } - assert await conversation.told() == { - "type": "tool_update", - "tool_call_id": "call-2", - "text": "One moment.", - "verbatim": True, - } - assert await conversation.told() == { - "type": "tool_result", - "tool_call_id": "call-2", - "output": "a table at 7:30", - } - async def test_a_held_call_goes_on_once_the_router_holds_it( self, conversation: Conversation, model: stream.STS ): diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/api/default/get_agent_stats.py b/plugins/stream/vision_agents/plugins/stream/_generated/api/default/get_agent_stats.py index 73de059253..afe3863f8b 100644 --- a/plugins/stream/vision_agents/plugins/stream/_generated/api/default/get_agent_stats.py +++ b/plugins/stream/vision_agents/plugins/stream/_generated/api/default/get_agent_stats.py @@ -8,13 +8,14 @@ from ...client import AuthenticatedClient, Client from ...models.agent_stats import AgentStats from ...models.error_response import ErrorResponse -from ...types import UNSET, Response +from ...types import UNSET, Response, Unset def _get_kwargs( *, from_: datetime.datetime, to: datetime.datetime, + agent: str | Unset = UNSET, ) -> dict[str, Any]: params: dict[str, Any] = {} @@ -25,6 +26,8 @@ def _get_kwargs( json_to = to.isoformat() params["to"] = json_to + params["agent"] = agent + params = {k: v for k, v in params.items() if v is not UNSET and v is not None} _kwargs: dict[str, Any] = { @@ -64,6 +67,11 @@ def _parse_response( return response_403 + if response.status_code == 404: + response_404 = ErrorResponse.from_dict(response.json()) + + return response_404 + if response.status_code == 500: response_500 = ErrorResponse.from_dict(response.json()) @@ -96,17 +104,23 @@ def sync_detailed( client: AuthenticatedClient | Client, from_: datetime.datetime, to: datetime.datetime, + agent: str | Unset = UNSET, ) -> Response[AgentStats | ErrorResponse]: """How the calling customer's agents ran, across all of them - The most agents that were running at once, and which tools they called: how often, how much each - call put in front of the model and how long it took. + The most agents that were running at once, and which tools they called: how often, in how many + sessions and after how many turns, how much each call put in front of the model, what the tool's + definition costs every turn that carries it and how long a call took. + With agent, the sessions, turns and tools are that agent config's alone; the peak is still every + agent's. One figure for the whole window rather than a row per bucket, because neither a peak nor a percentile can be summed out of smaller ones. Args: from_ (datetime.datetime): Start of the window, inclusive. to (datetime.datetime): End of the window, exclusive. + agent (str | Unset): The id of an agent config: only its sessions are counted. Omitted is + every agent. Raises: errors.UnexpectedStatus: If the server returns an undocumented status code and Client.raise_on_unexpected_status is True. @@ -119,6 +133,7 @@ def sync_detailed( kwargs = _get_kwargs( from_=from_, to=to, + agent=agent, ) response = client.get_httpx_client().request( @@ -133,17 +148,23 @@ def sync( client: AuthenticatedClient | Client, from_: datetime.datetime, to: datetime.datetime, + agent: str | Unset = UNSET, ) -> AgentStats | ErrorResponse | None: """How the calling customer's agents ran, across all of them - The most agents that were running at once, and which tools they called: how often, how much each - call put in front of the model and how long it took. + The most agents that were running at once, and which tools they called: how often, in how many + sessions and after how many turns, how much each call put in front of the model, what the tool's + definition costs every turn that carries it and how long a call took. + With agent, the sessions, turns and tools are that agent config's alone; the peak is still every + agent's. One figure for the whole window rather than a row per bucket, because neither a peak nor a percentile can be summed out of smaller ones. Args: from_ (datetime.datetime): Start of the window, inclusive. to (datetime.datetime): End of the window, exclusive. + agent (str | Unset): The id of an agent config: only its sessions are counted. Omitted is + every agent. Raises: errors.UnexpectedStatus: If the server returns an undocumented status code and Client.raise_on_unexpected_status is True. @@ -157,6 +178,7 @@ def sync( client=client, from_=from_, to=to, + agent=agent, ).parsed @@ -165,17 +187,23 @@ async def asyncio_detailed( client: AuthenticatedClient | Client, from_: datetime.datetime, to: datetime.datetime, + agent: str | Unset = UNSET, ) -> Response[AgentStats | ErrorResponse]: """How the calling customer's agents ran, across all of them - The most agents that were running at once, and which tools they called: how often, how much each - call put in front of the model and how long it took. + The most agents that were running at once, and which tools they called: how often, in how many + sessions and after how many turns, how much each call put in front of the model, what the tool's + definition costs every turn that carries it and how long a call took. + With agent, the sessions, turns and tools are that agent config's alone; the peak is still every + agent's. One figure for the whole window rather than a row per bucket, because neither a peak nor a percentile can be summed out of smaller ones. Args: from_ (datetime.datetime): Start of the window, inclusive. to (datetime.datetime): End of the window, exclusive. + agent (str | Unset): The id of an agent config: only its sessions are counted. Omitted is + every agent. Raises: errors.UnexpectedStatus: If the server returns an undocumented status code and Client.raise_on_unexpected_status is True. @@ -188,6 +216,7 @@ async def asyncio_detailed( kwargs = _get_kwargs( from_=from_, to=to, + agent=agent, ) response = await client.get_async_httpx_client().request(**kwargs) @@ -200,17 +229,23 @@ async def asyncio( client: AuthenticatedClient | Client, from_: datetime.datetime, to: datetime.datetime, + agent: str | Unset = UNSET, ) -> AgentStats | ErrorResponse | None: """How the calling customer's agents ran, across all of them - The most agents that were running at once, and which tools they called: how often, how much each - call put in front of the model and how long it took. + The most agents that were running at once, and which tools they called: how often, in how many + sessions and after how many turns, how much each call put in front of the model, what the tool's + definition costs every turn that carries it and how long a call took. + With agent, the sessions, turns and tools are that agent config's alone; the peak is still every + agent's. One figure for the whole window rather than a row per bucket, because neither a peak nor a percentile can be summed out of smaller ones. Args: from_ (datetime.datetime): Start of the window, inclusive. to (datetime.datetime): End of the window, exclusive. + agent (str | Unset): The id of an agent config: only its sessions are counted. Omitted is + every agent. Raises: errors.UnexpectedStatus: If the server returns an undocumented status code and Client.raise_on_unexpected_status is True. @@ -225,5 +260,6 @@ async def asyncio( client=client, from_=from_, to=to, + agent=agent, ) ).parsed diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/api/default/watch_session.py b/plugins/stream/vision_agents/plugins/stream/_generated/api/default/watch_session.py index f91e79bbbe..083b9d8efb 100644 --- a/plugins/stream/vision_agents/plugins/stream/_generated/api/default/watch_session.py +++ b/plugins/stream/vision_agents/plugins/stream/_generated/api/default/watch_session.py @@ -122,7 +122,8 @@ def sync_detailed( of it; it follows the utterance's `speaking`), `turn`, `decision`, `delegated`, `task_settled` (files lists what the work's code handed back, each a name, mime_type, url and size, uploaded to a persistent conversation's channel and attached to the reply), `task_cancelled`, `tool_call`, - `tool_held`, `tool_approval_ignored`, `tool_updated`, `tool_ran`, `transferred`, `pressed`, + `tool_held`, `tool_approval_ignored`, `tool_started`, `tool_updated`, `tool_ran`, + `tool_call.started`, `tool_call.progress`, `tool_call.finished`, `transferred`, `pressed`, `looked_up`, `backchannel`, `interrupted` (turn_id; with synthesis_id, heard_ms, message_start and message_end, the utterance being heard when it was cut, how many milliseconds of it had been, and where all it had been sent is in the reply's message, when any of it had), `overlap_decided`, @@ -176,6 +177,11 @@ def sync_detailed( tool_call_id, tool, turn_id and started_at, and pre_speech when the tool's connector binding sets one in its policy; tool_ran also includes tool_call_id, and files when the tool was run_code, as on task_settled. + `tool_call.started`, `tool_call.progress` and `tool_call.finished` (`ToolCallState`) are a call's + state as a client shows it, whoever runs the call: tool_call_id, tool, turn_id, status (in_progress, + waiting for a person's approval, success, failed or cancelled), progress (the last `tool_update`, + the binding's pre_speech or the approval question), started_at, duration_ms, approved_at, + rejected_at and error. Each carries the whole state: keep the last per tool_call_id. A respond command carrying request_id emits command_accepted with a nested command receipt (request_id, user_message_id, assistant_message_id, state, duplicate). Personal persistent text sessions require this ID. A retry with the same text and images returns the existing IDs without @@ -336,7 +342,8 @@ def sync( of it; it follows the utterance's `speaking`), `turn`, `decision`, `delegated`, `task_settled` (files lists what the work's code handed back, each a name, mime_type, url and size, uploaded to a persistent conversation's channel and attached to the reply), `task_cancelled`, `tool_call`, - `tool_held`, `tool_approval_ignored`, `tool_updated`, `tool_ran`, `transferred`, `pressed`, + `tool_held`, `tool_approval_ignored`, `tool_started`, `tool_updated`, `tool_ran`, + `tool_call.started`, `tool_call.progress`, `tool_call.finished`, `transferred`, `pressed`, `looked_up`, `backchannel`, `interrupted` (turn_id; with synthesis_id, heard_ms, message_start and message_end, the utterance being heard when it was cut, how many milliseconds of it had been, and where all it had been sent is in the reply's message, when any of it had), `overlap_decided`, @@ -390,6 +397,11 @@ def sync( tool_call_id, tool, turn_id and started_at, and pre_speech when the tool's connector binding sets one in its policy; tool_ran also includes tool_call_id, and files when the tool was run_code, as on task_settled. + `tool_call.started`, `tool_call.progress` and `tool_call.finished` (`ToolCallState`) are a call's + state as a client shows it, whoever runs the call: tool_call_id, tool, turn_id, status (in_progress, + waiting for a person's approval, success, failed or cancelled), progress (the last `tool_update`, + the binding's pre_speech or the approval question), started_at, duration_ms, approved_at, + rejected_at and error. Each carries the whole state: keep the last per tool_call_id. A respond command carrying request_id emits command_accepted with a nested command receipt (request_id, user_message_id, assistant_message_id, state, duplicate). Personal persistent text sessions require this ID. A retry with the same text and images returns the existing IDs without @@ -545,7 +557,8 @@ async def asyncio_detailed( of it; it follows the utterance's `speaking`), `turn`, `decision`, `delegated`, `task_settled` (files lists what the work's code handed back, each a name, mime_type, url and size, uploaded to a persistent conversation's channel and attached to the reply), `task_cancelled`, `tool_call`, - `tool_held`, `tool_approval_ignored`, `tool_updated`, `tool_ran`, `transferred`, `pressed`, + `tool_held`, `tool_approval_ignored`, `tool_started`, `tool_updated`, `tool_ran`, + `tool_call.started`, `tool_call.progress`, `tool_call.finished`, `transferred`, `pressed`, `looked_up`, `backchannel`, `interrupted` (turn_id; with synthesis_id, heard_ms, message_start and message_end, the utterance being heard when it was cut, how many milliseconds of it had been, and where all it had been sent is in the reply's message, when any of it had), `overlap_decided`, @@ -599,6 +612,11 @@ async def asyncio_detailed( tool_call_id, tool, turn_id and started_at, and pre_speech when the tool's connector binding sets one in its policy; tool_ran also includes tool_call_id, and files when the tool was run_code, as on task_settled. + `tool_call.started`, `tool_call.progress` and `tool_call.finished` (`ToolCallState`) are a call's + state as a client shows it, whoever runs the call: tool_call_id, tool, turn_id, status (in_progress, + waiting for a person's approval, success, failed or cancelled), progress (the last `tool_update`, + the binding's pre_speech or the approval question), started_at, duration_ms, approved_at, + rejected_at and error. Each carries the whole state: keep the last per tool_call_id. A respond command carrying request_id emits command_accepted with a nested command receipt (request_id, user_message_id, assistant_message_id, state, duplicate). Personal persistent text sessions require this ID. A retry with the same text and images returns the existing IDs without @@ -757,7 +775,8 @@ async def asyncio( of it; it follows the utterance's `speaking`), `turn`, `decision`, `delegated`, `task_settled` (files lists what the work's code handed back, each a name, mime_type, url and size, uploaded to a persistent conversation's channel and attached to the reply), `task_cancelled`, `tool_call`, - `tool_held`, `tool_approval_ignored`, `tool_updated`, `tool_ran`, `transferred`, `pressed`, + `tool_held`, `tool_approval_ignored`, `tool_started`, `tool_updated`, `tool_ran`, + `tool_call.started`, `tool_call.progress`, `tool_call.finished`, `transferred`, `pressed`, `looked_up`, `backchannel`, `interrupted` (turn_id; with synthesis_id, heard_ms, message_start and message_end, the utterance being heard when it was cut, how many milliseconds of it had been, and where all it had been sent is in the reply's message, when any of it had), `overlap_decided`, @@ -811,6 +830,11 @@ async def asyncio( tool_call_id, tool, turn_id and started_at, and pre_speech when the tool's connector binding sets one in its policy; tool_ran also includes tool_call_id, and files when the tool was run_code, as on task_settled. + `tool_call.started`, `tool_call.progress` and `tool_call.finished` (`ToolCallState`) are a call's + state as a client shows it, whoever runs the call: tool_call_id, tool, turn_id, status (in_progress, + waiting for a person's approval, success, failed or cancelled), progress (the last `tool_update`, + the binding's pre_speech or the approval question), started_at, duration_ms, approved_at, + rejected_at and error. Each carries the whole state: keep the last per tool_call_id. A respond command carrying request_id emits command_accepted with a nested command receipt (request_id, user_message_id, assistant_message_id, state, duplicate). Personal persistent text sessions require this ID. A retry with the same text and images returns the existing IDs without diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/__init__.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/__init__.py index 2116595fba..d3d4c0d8f1 100644 --- a/plugins/stream/vision_agents/plugins/stream/_generated/models/__init__.py +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/__init__.py @@ -56,7 +56,6 @@ from .agent_role import AgentRole from .agent_scheduling import AgentScheduling from .agent_stats import AgentStats -from .agent_tools import AgentTools from .agent_user import AgentUser from .agent_user_request import AgentUserRequest from .analyze_reviews_request import AnalyzeReviewsRequest @@ -596,6 +595,11 @@ from .tool_approval_answer import ToolApprovalAnswer from .tool_approval_command import ToolApprovalCommand from .tool_approval_command_type import ToolApprovalCommandType +from .tool_call_frame_type import ToolCallFrameType +from .tool_call_state import ToolCallState +from .tool_call_status import ToolCallStatus +from .tool_group import ToolGroup +from .tool_loading import ToolLoading from .tool_result_command import ToolResultCommand from .tool_result_command_type import ToolResultCommandType from .tool_stats import ToolStats @@ -716,7 +720,6 @@ "AgentRole", "AgentScheduling", "AgentStats", - "AgentTools", "AgentUser", "AgentUserRequest", "AnalyzeReviewsRequest", @@ -1250,6 +1253,11 @@ "ToolApprovalAnswer", "ToolApprovalCommand", "ToolApprovalCommandType", + "ToolCallFrameType", + "ToolCallState", + "ToolCallStatus", + "ToolGroup", + "ToolLoading", "ToolResultCommand", "ToolResultCommandType", "ToolStats", diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_config.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_config.py index 228fc38f37..131f3615cf 100644 --- a/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_config.py +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_config.py @@ -24,7 +24,6 @@ from ..models.agent_dispatch import AgentDispatch from ..models.agent_review import AgentReview from ..models.agent_scheduling import AgentScheduling - from ..models.agent_tools import AgentTools from ..models.child_agent import ChildAgent from ..models.floor_control import FloorControl from ..models.greeting import Greeting @@ -32,6 +31,7 @@ from ..models.resume_history import ResumeHistory from ..models.sandbox_options import SandboxOptions from ..models.session_video import SessionVideo + from ..models.tool_group import ToolGroup T = TypeVar("T", bound="AgentConfig") @@ -105,8 +105,8 @@ class AgentConfig: sync_hash (str | Unset): Fingerprint of the last directory synced onto this config. Empty if it was never synced from a directory. tags (AgentConfigTags | Unset): - tools (AgentTools | Unset): How an agent is offered its tools, and how long what they were given and gave back - is kept. + tools (list[ToolGroup] | Unset): The functions an SDK provides that the agent may offer, in named groups. Empty + when there are none, and then the agent offers no function. tts (str | Unset): video (SessionVideo | Unset): visible_tools (list[str] | Unset): @@ -156,7 +156,7 @@ class AgentConfig: subagent: str | Unset = UNSET sync_hash: str | Unset = UNSET tags: AgentConfigTags | Unset = UNSET - tools: AgentTools | Unset = UNSET + tools: list[ToolGroup] | Unset = UNSET tts: str | Unset = UNSET video: SessionVideo | Unset = UNSET visible_tools: list[str] | Unset = UNSET @@ -277,9 +277,12 @@ def to_dict(self) -> dict[str, Any]: if not isinstance(self.tags, Unset): tags = self.tags.to_dict() - tools: dict[str, Any] | Unset = UNSET + tools: list[dict[str, Any]] | Unset = UNSET if not isinstance(self.tools, Unset): - tools = self.tools.to_dict() + tools = [] + for tools_item_data in self.tools: + tools_item = tools_item_data.to_dict() + tools.append(tools_item) tts = self.tts @@ -414,7 +417,6 @@ def from_dict(cls, src_dict: Mapping[str, Any]) -> Self: from ..models.agent_dispatch import AgentDispatch from ..models.agent_review import AgentReview from ..models.agent_scheduling import AgentScheduling - from ..models.agent_tools import AgentTools from ..models.child_agent import ChildAgent from ..models.floor_control import FloorControl from ..models.greeting import Greeting @@ -422,6 +424,7 @@ def from_dict(cls, src_dict: Mapping[str, Any]) -> Self: from ..models.resume_history import ResumeHistory from ..models.sandbox_options import SandboxOptions from ..models.session_video import SessionVideo + from ..models.tool_group import ToolGroup d = dict(src_dict) created_at = datetime.datetime.fromisoformat(d.pop("created_at")) @@ -577,11 +580,13 @@ def from_dict(cls, src_dict: Mapping[str, Any]) -> Self: tags = AgentConfigTags.from_dict(_tags) _tools = d.pop("tools", UNSET) - tools: AgentTools | Unset - if isinstance(_tools, Unset): - tools = UNSET - else: - tools = AgentTools.from_dict(_tools) + tools: list[ToolGroup] | Unset = UNSET + if _tools is not UNSET: + tools = [] + for tools_item_data in _tools: + tools_item = ToolGroup.from_dict(tools_item_data) + + tools.append(tools_item) tts = d.pop("tts", UNSET) diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_config_patch.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_config_patch.py index 969da5ceb8..73845445e4 100644 --- a/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_config_patch.py +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_config_patch.py @@ -31,13 +31,13 @@ from ..models.agent_dispatch import AgentDispatch from ..models.agent_review import AgentReview from ..models.agent_scheduling import AgentScheduling - from ..models.agent_tools import AgentTools from ..models.child_agent import ChildAgent from ..models.floor_control import FloorControl from ..models.greeting import Greeting from ..models.resume_history import ResumeHistory from ..models.sandbox_options import SandboxOptions from ..models.session_video import SessionVideo + from ..models.tool_group import ToolGroup T = TypeVar("T", bound="AgentConfigPatch") @@ -116,8 +116,7 @@ class AgentConfigPatch: stt (str | Unset): subagent (str | Unset): Only a voice agent names one. Switching an agent to text drops it. tags (AgentConfigPatchTags | Unset): - tools (AgentTools | Unset): How an agent is offered its tools, and how long what they were given and gave back - is kept. + tools (list[ToolGroup] | Unset): Replaces the tool groups whole; an empty list removes them all. tts (str | Unset): video (SessionVideo | Unset): visible_tools (list[str] | Unset): Tools whose steps end users see on a persistent conversation's replies, as @@ -165,7 +164,7 @@ class AgentConfigPatch: stt: str | Unset = UNSET subagent: str | Unset = UNSET tags: AgentConfigPatchTags | Unset = UNSET - tools: AgentTools | Unset = UNSET + tools: list[ToolGroup] | Unset = UNSET tts: str | Unset = UNSET video: SessionVideo | Unset = UNSET visible_tools: list[str] | Unset = UNSET @@ -301,9 +300,12 @@ def to_dict(self) -> dict[str, Any]: if not isinstance(self.tags, Unset): tags = self.tags.to_dict() - tools: dict[str, Any] | Unset = UNSET + tools: list[dict[str, Any]] | Unset = UNSET if not isinstance(self.tools, Unset): - tools = self.tools.to_dict() + tools = [] + for tools_item_data in self.tools: + tools_item = tools_item_data.to_dict() + tools.append(tools_item) tts = self.tts @@ -446,13 +448,13 @@ def from_dict(cls, src_dict: Mapping[str, Any]) -> Self: from ..models.agent_dispatch import AgentDispatch from ..models.agent_review import AgentReview from ..models.agent_scheduling import AgentScheduling - from ..models.agent_tools import AgentTools from ..models.child_agent import ChildAgent from ..models.floor_control import FloorControl from ..models.greeting import Greeting from ..models.resume_history import ResumeHistory from ..models.sandbox_options import SandboxOptions from ..models.session_video import SessionVideo + from ..models.tool_group import ToolGroup d = dict(src_dict) _calls = d.pop("calls", UNSET) @@ -641,11 +643,13 @@ def _parse_plugins_item( tags = AgentConfigPatchTags.from_dict(_tags) _tools = d.pop("tools", UNSET) - tools: AgentTools | Unset - if isinstance(_tools, Unset): - tools = UNSET - else: - tools = AgentTools.from_dict(_tools) + tools: list[ToolGroup] | Unset = UNSET + if _tools is not UNSET: + tools = [] + for tools_item_data in _tools: + tools_item = ToolGroup.from_dict(tools_item_data) + + tools.append(tools_item) tts = d.pop("tts", UNSET) diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_config_request.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_config_request.py index 978d7e6a5c..acd618aa60 100644 --- a/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_config_request.py +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_config_request.py @@ -32,13 +32,13 @@ from ..models.agent_dispatch import AgentDispatch from ..models.agent_review import AgentReview from ..models.agent_scheduling import AgentScheduling - from ..models.agent_tools import AgentTools from ..models.child_agent import ChildAgent from ..models.floor_control import FloorControl from ..models.greeting import Greeting from ..models.resume_history import ResumeHistory from ..models.sandbox_options import SandboxOptions from ..models.session_video import SessionVideo + from ..models.tool_group import ToolGroup T = TypeVar("T", bound="AgentConfigRequest") @@ -133,8 +133,13 @@ class AgentConfigRequest: and a config naming another team tag is refused. A config tagged draft_of, naming the config it copies, is a test copy: it answers no message on a channel connection and subscribes to no event, which stay with the live config, and it may bind a channel connection another config binds. - tools (AgentTools | Unset): How an agent is offered its tools, and how long what they were given and gave back - is kept. + tools (list[ToolGroup] | Unset): The functions an SDK provides that the agent may offer, in named groups, each + with how it is loaded. A function an SDK registers, declares on a session or hosts in a worker is offered only + when a group names it: a session declaring one that none names is refused (400 tool_not_configured), and a + worker's is left out. The router's own tools, connector tools and other agents are not named here. Omitted or + null on an update, the groups stored stay; an empty list removes them all. The mapping this field once was, with + progressive and output_retention, is refused: loading summary and output_retention on a group or a connector + binding replaced them. tts (str | Unset): video (SessionVideo | Unset): visible_tools (list[str] | Unset): Tools whose steps end users see on a persistent conversation's replies, as @@ -209,7 +214,7 @@ class AgentConfigRequest: stt: str | Unset = UNSET subagent: str | Unset = UNSET tags: AgentConfigRequestTags | Unset = UNSET - tools: AgentTools | Unset = UNSET + tools: list[ToolGroup] | Unset = UNSET tts: str | Unset = UNSET video: SessionVideo | Unset = UNSET visible_tools: list[str] | Unset = UNSET @@ -346,9 +351,12 @@ def to_dict(self) -> dict[str, Any]: if not isinstance(self.tags, Unset): tags = self.tags.to_dict() - tools: dict[str, Any] | Unset = UNSET + tools: list[dict[str, Any]] | Unset = UNSET if not isinstance(self.tools, Unset): - tools = self.tools.to_dict() + tools = [] + for tools_item_data in self.tools: + tools_item = tools_item_data.to_dict() + tools.append(tools_item) tts = self.tts @@ -493,13 +501,13 @@ def from_dict(cls, src_dict: Mapping[str, Any]) -> Self: from ..models.agent_dispatch import AgentDispatch from ..models.agent_review import AgentReview from ..models.agent_scheduling import AgentScheduling - from ..models.agent_tools import AgentTools from ..models.child_agent import ChildAgent from ..models.floor_control import FloorControl from ..models.greeting import Greeting from ..models.resume_history import ResumeHistory from ..models.sandbox_options import SandboxOptions from ..models.session_video import SessionVideo + from ..models.tool_group import ToolGroup d = dict(src_dict) name = d.pop("name") @@ -688,11 +696,13 @@ def _parse_plugins_item( tags = AgentConfigRequestTags.from_dict(_tags) _tools = d.pop("tools", UNSET) - tools: AgentTools | Unset - if isinstance(_tools, Unset): - tools = UNSET - else: - tools = AgentTools.from_dict(_tools) + tools: list[ToolGroup] | Unset = UNSET + if _tools is not UNSET: + tools = [] + for tools_item_data in _tools: + tools_item = ToolGroup.from_dict(tools_item_data) + + tools.append(tools_item) tts = d.pop("tts", UNSET) diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_connector_binding.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_connector_binding.py index de4d9d908f..a0edef9ec5 100644 --- a/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_connector_binding.py +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_connector_binding.py @@ -7,6 +7,7 @@ from attrs import field as _attrs_field from typing_extensions import Self +from ..models.tool_loading import ToolLoading from ..types import UNSET, Unset if TYPE_CHECKING: @@ -64,6 +65,15 @@ class AgentConnectorBinding: lists that matches is pinned on that connection the first time, at the digest listed then, and a later session offers it only while that digest still matches. An empty list is refused. At most 128, the same cap as tools, which is also the most one session offers when more match. + loading (ToolLoading | Unset): How a tool is offered to the model. full sends its whole description and input + schema from the start. summary sends the first line of its description and its arguments without their + descriptions, and the first call returns the full description and schema instead of running it: one more model + turn for each tool a conversation uses, for less context on every turn. deferred offers nothing until the model + needs it: an agent with a deferred tool is offered search_tool, which finds deferred tools by what they do, and + load_tool, which returns their full descriptions and offers them from the model's next step on. + output_retention (str | Unset): How long the agent's sessions keep what the binding's tools were given and gave + back on the turns they record, as a tool group's output_retention. Omitted or empty, it is kept as long as the + session. Example: 30d. """ connection: AgentConnectorSelection @@ -77,6 +87,8 @@ class AgentConnectorBinding: sandbox: bool | Unset = UNSET timeout_ms: int | Unset = UNSET tool_patterns: list[str] | Unset = UNSET + loading: ToolLoading | Unset = UNSET + output_retention: str | Unset = UNSET additional_properties: dict[str, Any] = _attrs_field(init=False, factory=dict) def to_dict(self) -> dict[str, Any]: @@ -116,6 +128,12 @@ def to_dict(self) -> dict[str, Any]: if not isinstance(self.tool_patterns, Unset): tool_patterns = self.tool_patterns + loading: str | Unset = UNSET + if not isinstance(self.loading, Unset): + loading = self.loading.value + + output_retention = self.output_retention + field_dict: dict[str, Any] = {} field_dict.update(self.additional_properties) field_dict.update( @@ -140,6 +158,10 @@ def to_dict(self) -> dict[str, Any]: field_dict["timeout_ms"] = timeout_ms if tool_patterns is not UNSET: field_dict["tool_patterns"] = tool_patterns + if loading is not UNSET: + field_dict["loading"] = loading + if output_retention is not UNSET: + field_dict["output_retention"] = output_retention return field_dict @@ -190,6 +212,15 @@ def from_dict(cls, src_dict: Mapping[str, Any]) -> Self: tool_patterns = cast(list[str], d.pop("tool_patterns", UNSET)) + _loading = d.pop("loading", UNSET) + loading: ToolLoading | Unset + if isinstance(_loading, Unset): + loading = UNSET + else: + loading = ToolLoading._decode(_loading) + + output_retention = d.pop("output_retention", UNSET) + agent_connector_binding = cls( connection=connection, connector_id=connector_id, @@ -202,6 +233,8 @@ def from_dict(cls, src_dict: Mapping[str, Any]) -> Self: sandbox=sandbox, timeout_ms=timeout_ms, tool_patterns=tool_patterns, + loading=loading, + output_retention=output_retention, ) agent_connector_binding.additional_properties = d diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_stats.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_stats.py index 8575032ff7..4a089b90b5 100644 --- a/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_stats.py +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_stats.py @@ -25,12 +25,17 @@ class AgentStats: still open counts up to now. tool_calls (ToolStats): tools (list[ToolStats]): One row per tool, most called first. + avg_turns (float | None): Turns a session took in the window, on average. A turn is a reply to the person, not a + rewound one or a delegated task's. Null when no session took one. + sessions (int): Sessions that took a turn in the window: what a tool's session_share is out of. """ peak_at: datetime.datetime | None peak_concurrent_sessions: int tool_calls: ToolStats tools: list[ToolStats] + avg_turns: float | None + sessions: int additional_properties: dict[str, Any] = _attrs_field(init=False, factory=dict) def to_dict(self) -> dict[str, Any]: @@ -49,6 +54,11 @@ def to_dict(self) -> dict[str, Any]: tools_item = tools_item_data.to_dict() tools.append(tools_item) + avg_turns: float | None + avg_turns = self.avg_turns + + sessions = self.sessions + field_dict: dict[str, Any] = {} field_dict.update(self.additional_properties) field_dict.update( @@ -57,6 +67,8 @@ def to_dict(self) -> dict[str, Any]: "peak_concurrent_sessions": peak_concurrent_sessions, "tool_calls": tool_calls, "tools": tools, + "avg_turns": avg_turns, + "sessions": sessions, } ) @@ -94,11 +106,22 @@ def _parse_peak_at(data: object) -> datetime.datetime | None: tools.append(tools_item) + def _parse_avg_turns(data: object) -> float | None: + if data is None: + return data + return cast(float | None, data) + + avg_turns = _parse_avg_turns(d.pop("avg_turns")) + + sessions = d.pop("sessions") + agent_stats = cls( peak_at=peak_at, peak_concurrent_sessions=peak_concurrent_sessions, tool_calls=tool_calls, tools=tools, + avg_turns=avg_turns, + sessions=sessions, ) agent_stats.additional_properties = d diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_tools.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_tools.py deleted file mode 100644 index 9d5df9ee22..0000000000 --- a/plugins/stream/vision_agents/plugins/stream/_generated/models/agent_tools.py +++ /dev/null @@ -1,82 +0,0 @@ -from __future__ import annotations - -from collections.abc import Mapping -from typing import Any, TypeVar - -from attrs import define as _attrs_define -from attrs import field as _attrs_field -from typing_extensions import Self - -from ..types import UNSET, Unset - -T = TypeVar("T", bound="AgentTools") - - -@_attrs_define -class AgentTools: - """How an agent is offered its tools, and how long what they were given and gave back is kept. - - Attributes: - output_retention (str | Unset): How long this agent's sessions keep what any of its tools were given and gave - back on the turns they record: a tool's arguments, its result, a search's query and the wording of a tool's - error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long - after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id - and whether it failed. Left out on a create, or empty, it is kept as long as the session; left out on an update, - the stored setting stays, and an empty string removes it. The app's and the organization's policies - (tool_output_retention) can shorten it and never lengthen it, and the shortest applies. It applies to tool calls - recorded after it is set. Example: 30d. - progressive (bool | Unset): Offer each tool by the first line of its description, with its arguments' - descriptions left out, and have the first call to a tool return its full description and input schema instead of - running it. It saves context on an agent with many tools, at the cost of one more model turn for each tool a - conversation uses. Off by default. Left out on an update, the stored setting stays. - """ - - output_retention: str | Unset = UNSET - progressive: bool | Unset = UNSET - additional_properties: dict[str, Any] = _attrs_field(init=False, factory=dict) - - def to_dict(self) -> dict[str, Any]: - output_retention = self.output_retention - - progressive = self.progressive - - field_dict: dict[str, Any] = {} - field_dict.update(self.additional_properties) - field_dict.update({}) - if output_retention is not UNSET: - field_dict["output_retention"] = output_retention - if progressive is not UNSET: - field_dict["progressive"] = progressive - - return field_dict - - @classmethod - def from_dict(cls, src_dict: Mapping[str, Any]) -> Self: - d = dict(src_dict) - output_retention = d.pop("output_retention", UNSET) - - progressive = d.pop("progressive", UNSET) - - agent_tools = cls( - output_retention=output_retention, - progressive=progressive, - ) - - agent_tools.additional_properties = d - return agent_tools - - @property - def additional_keys(self) -> list[str]: - return list(self.additional_properties.keys()) - - def __getitem__(self, key: str) -> Any: - return self.additional_properties[key] - - def __setitem__(self, key: str, value: Any) -> None: - self.additional_properties[key] = value - - def __delitem__(self, key: str) -> None: - del self.additional_properties[key] - - def __contains__(self, key: str) -> bool: - return key in self.additional_properties diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/error_detail.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/error_detail.py index 54d5f3e7b4..130d10c210 100644 --- a/plugins/stream/vision_agents/plugins/stream/_generated/models/error_detail.py +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/error_detail.py @@ -30,62 +30,64 @@ class ErrorDetail: failed or canceled. Opening the session is refused, or, when the status came while the agent was joining the call, the session is ended at once and kept as ended), no_oauth_client (a 400: a consent found no OAuth client for the connector, and the message names each way the connector lets the app give it one), invalid_config (a - 400: an agent config a run names does not start a session), insufficient_credits (a 402: the organization's - credit is spent, so work that costs money is refused until credit is added; a model that costs nothing is still - served), system_limit_reached (a 402: the organization's spend hard limit for the day or the month is reached, - whatever its credit, and the message names the interval, the amount and when it resets), spend_limit_reached (a - 402: one of the organization's own spend limits is spent, on the organization, an app, a key, a usage category - or a team, and the message says which and when it resets, so retrying before then is refused again), - user_limit_reached (a 429: a spend limit on the one end user the request is for is spent, which stops that user - and no one else, and Retry-After says when its window ends), spend_check_unavailable (a 503: the router could - not share what the organization is spending with the other routers for a while, so work that costs money is - refused until it can; a model that costs nothing is still served, and Retry-After says to try again in a few - seconds), budget_moved (a 400: a policy sent a budget, which spend limits replaced), limit_above_parent (a 400: - a spend limit is above a limit of its parent in the same interval, and the message names the parent's limit), - limit_exists (a 409: the subject already has a limit at that level and interval, and the message names its id), - no_limit (a 404: the subject has no spend limit to reset or grant past), use_team (a 400: a category starting - team: was sent, where the team field names a team), invalid_expiry (a 400: a grant's expires_at is not in the - future, or is more than 366 days away), too_many_grants (a 409: the organization has 20 live grants, as many as - it may), tier_limit_reached (the organization's tier limits refuse the work, and limit says which: a 403 for a - call or a transfer, to a country the tier does not reach, past its minutes or recipients of the day, or past its - calls at once; a 400 for an agent config whose sandbox is bigger or more open than the organization's sandbox - limits allow. The message names the tier and the limit), account_suspended (a 403: the organization is - suspended, and every request is refused, one for a model that costs nothing included, until it is restored; the - message says to contact support), concurrency_limited (a 429: the app has as many runs in flight as it may, and - Retry-After says when to try again), model_failed (a 503: the model failed or gave no answer), prompt_injection - (a 400: the prompt injection screen refused the input of a run or a tool result it read, which retrying an - unchanged input does not fix), image_unavailable (a 400: an image a run was handed could not be fetched or is - not a JPEG, PNG, WebP or GIF, which retrying the same image does not fix), document_too_large (a 413: a file - sent with a response has more pages or text than one may, alone or with the files the conversation already - holds, and the message names the limit), document_unreadable (a 400: a file sent with a response could not be - read, such as one with a password, one that is damaged or one with no text in it, which retrying the same file - does not fix), invalid_usage_categories (a 400: the usage categories the request names, in the token's - usage_categories and team claims or in X-Stream-Usage-Categories, are more than 10, one is not 1 to 64 - characters of A-Z, a-z, 0-9, _ . : -, more than one names a team, or the team is over 59 characters), - memory_unavailable (a 503: the memory store could not delete what was asked, so nothing that waits on it was - deleted either, and the same request later can succeed), billing_not_enabled (a 404: this deployment bills - nobody, or the caller's app is in no organization it bills, so there is no account to read), - provider_takes_no_key (a 400: a key was put for a provider whose work is never routed on a customer's own key - here, and the message names the ones that are), campaign_locked (a 409: the campaign's state does not allow the - change, such as a launched campaign's sequence, or resuming one that is running), campaign_not_ready (a 400: a - launch found the campaign missing what it needs to ring anybody, and the message says what), consent_required (a - 400: a launch did not say the people on the campaign agreed to be called), branch_limit_reached (a 409: the - conversation has as many edit and regenerate points as Stream Chat keeps for it, 100, so a regenerate or an edit - is refused and the conversation is continued in a new chat; asking and reading are unaffected), branch_too_long - (a 409: the branch an activation would bring back is more than 100 messages to send again, so it is continued in - a new chat), branch_unavailable (a 410: the copies of a hidden branch were deleted, as a memory truncate deletes - them, so it can be neither read nor brought back), switch_unfinished (a 409: a regenerate, an edit or an - activation of the session failed after Stream Chat may have rewound, and finishing it still fails because Stream - Chat does not answer, so the session's questions, regenerates, edits and activations are refused meanwhile; try - again: each one tries to finish it first, so it clears once Stream Chat answers), and _not_found for - agent_config, call, campaign, channel_account, command, connection, grant, knowledge_document, knowledge_url, - limit, provider_key, router_config, scheduled_task, session, simulation, simulation_run, skill and voice. A - scheduled task also has invalid_schedule (a schedule that does not read: at and in both or neither, a date, time - or duration that does not parse, or until without repeat), schedule_in_past, schedule_too_far (a one-shot, or an - in, more than a year ahead), invalid_timezone, timezone_unknown (no zone was given and neither the user nor the - config has one), invalid_repeat, interval_too_short (a run moved to less than 15 minutes from now and earlier - than the run it replaces) and too_many_tasks (a 409: the user is at their active-task limit). More may be added, - so a client should expect one it does not know. + 400: an agent config a run names does not start a session), tool_not_configured (a 400: a session under an agent + config was declared or given a function that none of the config's tool groups names, and the message names it), + insufficient_credits (a 402: the organization's credit is spent, so work that costs money is refused until + credit is added; a model that costs nothing is still served), system_limit_reached (a 402: the organization's + spend hard limit for the day or the month is reached, whatever its credit, and the message names the interval, + the amount and when it resets), spend_limit_reached (a 402: one of the organization's own spend limits is spent, + on the organization, an app, a key, a usage category or a team, and the message says which and when it resets, + so retrying before then is refused again), user_limit_reached (a 429: a spend limit on the one end user the + request is for is spent, which stops that user and no one else, and Retry-After says when its window ends), + spend_check_unavailable (a 503: the router could not share what the organization is spending with the other + routers for a while, so work that costs money is refused until it can; a model that costs nothing is still + served, and Retry-After says to try again in a few seconds), budget_moved (a 400: a policy sent a budget, which + spend limits replaced), limit_above_parent (a 400: a spend limit is above a limit of its parent in the same + interval, and the message names the parent's limit), limit_exists (a 409: the subject already has a limit at + that level and interval, and the message names its id), no_limit (a 404: the subject has no spend limit to reset + or grant past), use_team (a 400: a category starting team: was sent, where the team field names a team), + invalid_expiry (a 400: a grant's expires_at is not in the future, or is more than 366 days away), + too_many_grants (a 409: the organization has 20 live grants, as many as it may), tier_limit_reached (the + organization's tier limits refuse the work, and limit says which: a 403 for a call or a transfer, to a country + the tier does not reach, past its minutes or recipients of the day, or past its calls at once; a 400 for an + agent config whose sandbox is bigger or more open than the organization's sandbox limits allow. The message + names the tier and the limit), account_suspended (a 403: the organization is suspended, and every request is + refused, one for a model that costs nothing included, until it is restored; the message says to contact + support), concurrency_limited (a 429: the app has as many runs in flight as it may, and Retry-After says when to + try again), model_failed (a 503: the model failed or gave no answer), prompt_injection (a 400: the prompt + injection screen refused the input of a run or a tool result it read, which retrying an unchanged input does not + fix), image_unavailable (a 400: an image a run was handed could not be fetched or is not a JPEG, PNG, WebP or + GIF, which retrying the same image does not fix), document_too_large (a 413: a file sent with a response has + more pages or text than one may, alone or with the files the conversation already holds, and the message names + the limit), document_unreadable (a 400: a file sent with a response could not be read, such as one with a + password, one that is damaged or one with no text in it, which retrying the same file does not fix), + invalid_usage_categories (a 400: the usage categories the request names, in the token's usage_categories and + team claims or in X-Stream-Usage-Categories, are more than 10, one is not 1 to 64 characters of A-Z, a-z, 0-9, _ + . : -, more than one names a team, or the team is over 59 characters), memory_unavailable (a 503: the memory + store could not delete what was asked, so nothing that waits on it was deleted either, and the same request + later can succeed), billing_not_enabled (a 404: this deployment bills nobody, or the caller's app is in no + organization it bills, so there is no account to read), provider_takes_no_key (a 400: a key was put for a + provider whose work is never routed on a customer's own key here, and the message names the ones that are), + campaign_locked (a 409: the campaign's state does not allow the change, such as a launched campaign's sequence, + or resuming one that is running), campaign_not_ready (a 400: a launch found the campaign missing what it needs + to ring anybody, and the message says what), consent_required (a 400: a launch did not say the people on the + campaign agreed to be called), branch_limit_reached (a 409: the conversation has as many edit and regenerate + points as Stream Chat keeps for it, 100, so a regenerate or an edit is refused and the conversation is continued + in a new chat; asking and reading are unaffected), branch_too_long (a 409: the branch an activation would bring + back is more than 100 messages to send again, so it is continued in a new chat), branch_unavailable (a 410: the + copies of a hidden branch were deleted, as a memory truncate deletes them, so it can be neither read nor brought + back), switch_unfinished (a 409: a regenerate, an edit or an activation of the session failed after Stream Chat + may have rewound, and finishing it still fails because Stream Chat does not answer, so the session's questions, + regenerates, edits and activations are refused meanwhile; try again: each one tries to finish it first, so it + clears once Stream Chat answers), and _not_found for agent_config, call, campaign, channel_account, + command, connection, grant, knowledge_document, knowledge_url, limit, provider_key, router_config, + scheduled_task, session, simulation, simulation_run, skill and voice. A scheduled task also has invalid_schedule + (a schedule that does not read: at and in both or neither, a date, time or duration that does not parse, or + until without repeat), schedule_in_past, schedule_too_far (a one-shot, or an in, more than a year ahead), + invalid_timezone, timezone_unknown (no zone was given and neither the user nor the config has one), + invalid_repeat, interval_too_short (a run moved to less than 15 minutes from now and earlier than the run it + replaces) and too_many_tasks (a 409: the user is at their active-task limit). More may be added, so a client + should expect one it does not know. doc_url (str): Where the code is explained. message (str): What went wrong, for a person to read. Its wording may change; branch on code. type_ (ErrorType): The kind of failure, which decides the status it is answered with: invalid_request 400, diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/offered_tool.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/offered_tool.py index caaa7be11f..6d19d3eabd 100644 --- a/plugins/stream/vision_agents/plugins/stream/_generated/models/offered_tool.py +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/offered_tool.py @@ -8,6 +8,7 @@ from typing_extensions import Self from ..models.offered_tool_source import OfferedToolSource +from ..models.tool_loading import ToolLoading from ..types import UNSET, Unset if TYPE_CHECKING: @@ -31,6 +32,12 @@ class OfferedTool: worker, hosted by a worker connected when the session opened; connector, opened by one of the agent's connector bindings; builtin, the router's own. A session's tools are fixed when it opens, so a worker that was not connected then leaves none of its tools here. Absent for a session kept before sources were recorded. + loading (ToolLoading | Unset): How a tool is offered to the model. full sends its whole description and input + schema from the start. summary sends the first line of its description and its arguments without their + descriptions, and the first call returns the full description and schema instead of running it: one more model + turn for each tool a conversation uses, for less context on every turn. deferred offers nothing until the model + needs it: an agent with a deferred tool is offered search_tool, which finds deferred tools by what they do, and + load_tool, which returns their full descriptions and offers them from the model's next step on. """ description: str @@ -38,6 +45,7 @@ class OfferedTool: tokens: int parameters: OfferedToolParameters | Unset = UNSET source: OfferedToolSource | Unset = UNSET + loading: ToolLoading | Unset = UNSET additional_properties: dict[str, Any] = _attrs_field(init=False, factory=dict) def to_dict(self) -> dict[str, Any]: @@ -55,6 +63,10 @@ def to_dict(self) -> dict[str, Any]: if not isinstance(self.source, Unset): source = self.source.value + loading: str | Unset = UNSET + if not isinstance(self.loading, Unset): + loading = self.loading.value + field_dict: dict[str, Any] = {} field_dict.update(self.additional_properties) field_dict.update( @@ -68,6 +80,8 @@ def to_dict(self) -> dict[str, Any]: field_dict["parameters"] = parameters if source is not UNSET: field_dict["source"] = source + if loading is not UNSET: + field_dict["loading"] = loading return field_dict @@ -96,12 +110,20 @@ def from_dict(cls, src_dict: Mapping[str, Any]) -> Self: else: source = OfferedToolSource._decode(_source) + _loading = d.pop("loading", UNSET) + loading: ToolLoading | Unset + if isinstance(_loading, Unset): + loading = UNSET + else: + loading = ToolLoading._decode(_loading) + offered_tool = cls( description=description, name=name, tokens=tokens, parameters=parameters, source=source, + loading=loading, ) offered_tool.additional_properties = d diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/policy.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/policy.py index ff7bbcd032..30ab8e3c1e 100644 --- a/plugins/stream/vision_agents/plugins/stream/_generated/models/policy.py +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/policy.py @@ -67,9 +67,10 @@ class Policy: records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed, so a session's items read in the same order and tool usage still counts the call. Left out, or empty, it - is kept as long as the session. The shortest of the organization's, the app's and the agent config's - tools.output_retention applies, so an agent can shorten it and never lengthen it. It applies to tool calls - recorded after it is set. Deleting a session deletes all of it at once. Example: 30d. + is kept as long as the session. The shortest of the organization's, the app's and the output_retention of the + agent config's tool group or connector binding the tool is in applies, so an agent can shorten it and never + lengthen it. It applies to tool calls recorded after it is set. Deleting a session deletes all of it at once. + Example: 30d. """ allowed_models: list[str] | Unset = UNSET diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/sync_agent_request.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/sync_agent_request.py index 0340031d38..6dfa8cd47a 100644 --- a/plugins/stream/vision_agents/plugins/stream/_generated/models/sync_agent_request.py +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/sync_agent_request.py @@ -20,7 +20,6 @@ from ..models.agent_connector_binding import AgentConnectorBinding from ..models.agent_dispatch import AgentDispatch from ..models.agent_scheduling import AgentScheduling - from ..models.agent_tools import AgentTools from ..models.child_agent import ChildAgent from ..models.floor_control import FloorControl from ..models.greeting import Greeting @@ -42,6 +41,7 @@ SyncAgentRequestPluginsItemType1, ) from ..models.sync_agent_request_tags import SyncAgentRequestTags + from ..models.tool_group import ToolGroup T = TypeVar("T", bound="SyncAgentRequest") @@ -136,8 +136,9 @@ class SyncAgentRequest: stt (str | Unset): subagent (str | Unset): Only a voice agent names one: a text agent runs everything on its llm. tags (SyncAgentRequestTags | Unset): - tools (AgentTools | Unset): How an agent is offered its tools, and how long what they were given and gave back - is kept. + tools (list[ToolGroup] | Unset): agent.yaml's tools: the functions an SDK provides that the agent may offer, in + named groups, as AgentConfigRequest's tools names them, with the same rules. Sent, they replace the stored + groups whole, an empty list removing them. Left out, the stored ones stay. tts (str | Unset): video (SessionVideo | Unset): voice (str | Unset): The voice the agent speaks in, as TtsOptions.voice takes it: a provider's own voice id, or @@ -192,7 +193,7 @@ class SyncAgentRequest: stt: str | Unset = UNSET subagent: str | Unset = UNSET tags: SyncAgentRequestTags | Unset = UNSET - tools: AgentTools | Unset = UNSET + tools: list[ToolGroup] | Unset = UNSET tts: str | Unset = UNSET video: SessionVideo | Unset = UNSET voice: str | Unset = UNSET @@ -354,9 +355,12 @@ def to_dict(self) -> dict[str, Any]: if not isinstance(self.tags, Unset): tags = self.tags.to_dict() - tools: dict[str, Any] | Unset = UNSET + tools: list[dict[str, Any]] | Unset = UNSET if not isinstance(self.tools, Unset): - tools = self.tools.to_dict() + tools = [] + for tools_item_data in self.tools: + tools_item = tools_item_data.to_dict() + tools.append(tools_item) tts = self.tts @@ -485,7 +489,6 @@ def from_dict(cls, src_dict: Mapping[str, Any]) -> Self: from ..models.agent_connector_binding import AgentConnectorBinding from ..models.agent_dispatch import AgentDispatch from ..models.agent_scheduling import AgentScheduling - from ..models.agent_tools import AgentTools from ..models.child_agent import ChildAgent from ..models.floor_control import FloorControl from ..models.greeting import Greeting @@ -507,6 +510,7 @@ def from_dict(cls, src_dict: Mapping[str, Any]) -> Self: SyncAgentRequestPluginsItemType1, ) from ..models.sync_agent_request_tags import SyncAgentRequestTags + from ..models.tool_group import ToolGroup d = dict(src_dict) hash_ = d.pop("hash") @@ -734,11 +738,13 @@ def _parse_plugins_item( tags = SyncAgentRequestTags.from_dict(_tags) _tools = d.pop("tools", UNSET) - tools: AgentTools | Unset - if isinstance(_tools, Unset): - tools = UNSET - else: - tools = AgentTools.from_dict(_tools) + tools: list[ToolGroup] | Unset = UNSET + if _tools is not UNSET: + tools = [] + for tools_item_data in _tools: + tools_item = ToolGroup.from_dict(tools_item_data) + + tools.append(tools_item) tts = d.pop("tts", UNSET) diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_call_frame_type.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_call_frame_type.py new file mode 100644 index 0000000000..f74fdfcfff --- /dev/null +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_call_frame_type.py @@ -0,0 +1,30 @@ +from enum import StrEnum + + +class ToolCallFrameType(StrEnum): + TOOL_CALL_FINISHED = "tool_call.finished" + TOOL_CALL_PROGRESS = "tool_call.progress" + TOOL_CALL_STARTED = "tool_call.started" + + def __str__(self) -> str: + return str(self.value) + + @classmethod + def _decode(cls, value: object) -> "ToolCallFrameType": + """Reads the value from an answer. A string this client does not list yet is kept as a member of its own, + so an answer from a newer router still reads, and is written back as it came.""" + try: + return cls(value) + except ValueError: + if not isinstance(value, str): + raise + member = str.__new__(cls, value) + member._name_ = value + member._value_ = value + return member + + def __reduce_ex__(self, protocol: object) -> object: + """Pickles a member this client does not list through _decode, as Enum's own reduce would call cls(value).""" + if self._value_ in type(self)._value2member_map_: + return super().__reduce_ex__(protocol) + return type(self)._decode, (self._value_,) diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_call_state.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_call_state.py new file mode 100644 index 0000000000..799f393ccd --- /dev/null +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_call_state.py @@ -0,0 +1,172 @@ +from __future__ import annotations + +import datetime +from collections.abc import Mapping +from typing import Any, TypeVar + +from attrs import define as _attrs_define +from attrs import field as _attrs_field +from typing_extensions import Self + +from ..models.tool_call_frame_type import ToolCallFrameType +from ..models.tool_call_status import ToolCallStatus +from ..types import UNSET, Unset + +T = TypeVar("T", bound="ToolCallState") + + +@_attrs_define +class ToolCallState: + """One change to a tool call's state. Every watcher is sent tool_call.started once, tool_call.progress for each change + while it runs or waits, and tool_call.finished once, carrying the call's whole state each time, so a client keeps + the last one it got per tool_call_id. + + Attributes: + duration_ms (int): From started_at to this change; on tool_call.finished, how long the call took. + started_at (datetime.datetime): + status (ToolCallStatus): in_progress while it runs, waiting while it is held for a person's approval, and + success, failed or cancelled once it finished. A call declined, or whose approval nobody gave in time, is + cancelled with rejected_at. + tool (str): + tool_call_id (str): The call, as tool_call, tool_started and tool_ran name it. + turn_id (str): + type_ (ToolCallFrameType): tool_call.started is the call's first frame, tool_call.progress one while it runs or + waits, and tool_call.finished its last. + approved_at (datetime.datetime | Unset): When a person allowed the call. + error (str | Unset): Why a failed call failed, or what a declined one shows, such as "Not allowed in time". + progress (str | Unset): The last thing said about the call while it runs, such as "Searching for flights": a + tool_update, the connector binding's pre_speech, or the question it waits on. + rejected_at (datetime.datetime | Unset): When a person declined the call, or its wait for them passed. + """ + + duration_ms: int + started_at: datetime.datetime + status: ToolCallStatus + tool: str + tool_call_id: str + turn_id: str + type_: ToolCallFrameType + approved_at: datetime.datetime | Unset = UNSET + error: str | Unset = UNSET + progress: str | Unset = UNSET + rejected_at: datetime.datetime | Unset = UNSET + additional_properties: dict[str, Any] = _attrs_field(init=False, factory=dict) + + def to_dict(self) -> dict[str, Any]: + duration_ms = self.duration_ms + + started_at = self.started_at.isoformat() + + status = self.status.value + + tool = self.tool + + tool_call_id = self.tool_call_id + + turn_id = self.turn_id + + type_ = self.type_.value + + approved_at: str | Unset = UNSET + if not isinstance(self.approved_at, Unset): + approved_at = self.approved_at.isoformat() + + error = self.error + + progress = self.progress + + rejected_at: str | Unset = UNSET + if not isinstance(self.rejected_at, Unset): + rejected_at = self.rejected_at.isoformat() + + field_dict: dict[str, Any] = {} + field_dict.update(self.additional_properties) + field_dict.update( + { + "duration_ms": duration_ms, + "started_at": started_at, + "status": status, + "tool": tool, + "tool_call_id": tool_call_id, + "turn_id": turn_id, + "type": type_, + } + ) + if approved_at is not UNSET: + field_dict["approved_at"] = approved_at + if error is not UNSET: + field_dict["error"] = error + if progress is not UNSET: + field_dict["progress"] = progress + if rejected_at is not UNSET: + field_dict["rejected_at"] = rejected_at + + return field_dict + + @classmethod + def from_dict(cls, src_dict: Mapping[str, Any]) -> Self: + d = dict(src_dict) + duration_ms = d.pop("duration_ms") + + started_at = datetime.datetime.fromisoformat(d.pop("started_at")) + + status = ToolCallStatus._decode(d.pop("status")) + + tool = d.pop("tool") + + tool_call_id = d.pop("tool_call_id") + + turn_id = d.pop("turn_id") + + type_ = ToolCallFrameType._decode(d.pop("type")) + + _approved_at = d.pop("approved_at", UNSET) + approved_at: datetime.datetime | Unset + if isinstance(_approved_at, Unset): + approved_at = UNSET + else: + approved_at = datetime.datetime.fromisoformat(_approved_at) + + error = d.pop("error", UNSET) + + progress = d.pop("progress", UNSET) + + _rejected_at = d.pop("rejected_at", UNSET) + rejected_at: datetime.datetime | Unset + if isinstance(_rejected_at, Unset): + rejected_at = UNSET + else: + rejected_at = datetime.datetime.fromisoformat(_rejected_at) + + tool_call_state = cls( + duration_ms=duration_ms, + started_at=started_at, + status=status, + tool=tool, + tool_call_id=tool_call_id, + turn_id=turn_id, + type_=type_, + approved_at=approved_at, + error=error, + progress=progress, + rejected_at=rejected_at, + ) + + tool_call_state.additional_properties = d + return tool_call_state + + @property + def additional_keys(self) -> list[str]: + return list(self.additional_properties.keys()) + + def __getitem__(self, key: str) -> Any: + return self.additional_properties[key] + + def __setitem__(self, key: str, value: Any) -> None: + self.additional_properties[key] = value + + def __delitem__(self, key: str) -> None: + del self.additional_properties[key] + + def __contains__(self, key: str) -> bool: + return key in self.additional_properties diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_call_status.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_call_status.py new file mode 100644 index 0000000000..2a1ce50eb4 --- /dev/null +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_call_status.py @@ -0,0 +1,32 @@ +from enum import StrEnum + + +class ToolCallStatus(StrEnum): + CANCELLED = "cancelled" + FAILED = "failed" + IN_PROGRESS = "in_progress" + SUCCESS = "success" + WAITING = "waiting" + + def __str__(self) -> str: + return str(self.value) + + @classmethod + def _decode(cls, value: object) -> "ToolCallStatus": + """Reads the value from an answer. A string this client does not list yet is kept as a member of its own, + so an answer from a newer router still reads, and is written back as it came.""" + try: + return cls(value) + except ValueError: + if not isinstance(value, str): + raise + member = str.__new__(cls, value) + member._name_ = value + member._value_ = value + return member + + def __reduce_ex__(self, protocol: object) -> object: + """Pickles a member this client does not list through _decode, as Enum's own reduce would call cls(value).""" + if self._value_ in type(self)._value2member_map_: + return super().__reduce_ex__(protocol) + return type(self)._decode, (self._value_,) diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_group.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_group.py new file mode 100644 index 0000000000..b7a99a3ae7 --- /dev/null +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_group.py @@ -0,0 +1,112 @@ +from __future__ import annotations + +from collections.abc import Mapping +from typing import Any, TypeVar, cast + +from attrs import define as _attrs_define +from attrs import field as _attrs_field +from typing_extensions import Self + +from ..models.tool_loading import ToolLoading +from ..types import UNSET, Unset + +T = TypeVar("T", bound="ToolGroup") + + +@_attrs_define +class ToolGroup: + """A named set of functions an SDK provides that the agent may offer, how they are loaded and how long what they were + given and gave back is kept. A function an SDK registers, declares on a session or hosts in a worker is offered only + when a group names it. + + Attributes: + name (str): What the group is called, unique within the config. + tools (list[str]): The functions' names, as the SDK registers them. A name sits in one group at most. + loading (ToolLoading | Unset): How a tool is offered to the model. full sends its whole description and input + schema from the start. summary sends the first line of its description and its arguments without their + descriptions, and the first call returns the full description and schema instead of running it: one more model + turn for each tool a conversation uses, for less context on every turn. deferred offers nothing until the model + needs it: an agent with a deferred tool is offered search_tool, which finds deferred tools by what they do, and + load_tool, which returns their full descriptions and offers them from the model's next step on. + output_retention (str | Unset): How long the agent's sessions keep what the group's tools were given and gave + back on the turns they record: a tool's arguments, its result and the wording of its error. none records none of + it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes + it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed. Left + out or empty, it is kept as long as the session. The app's and the organization's policies + (tool_output_retention) can shorten it and never lengthen it. It applies to tool calls recorded after it is set. + Example: 30d. + """ + + name: str + tools: list[str] + loading: ToolLoading | Unset = UNSET + output_retention: str | Unset = UNSET + additional_properties: dict[str, Any] = _attrs_field(init=False, factory=dict) + + def to_dict(self) -> dict[str, Any]: + name = self.name + + tools = self.tools + + loading: str | Unset = UNSET + if not isinstance(self.loading, Unset): + loading = self.loading.value + + output_retention = self.output_retention + + field_dict: dict[str, Any] = {} + field_dict.update(self.additional_properties) + field_dict.update( + { + "name": name, + "tools": tools, + } + ) + if loading is not UNSET: + field_dict["loading"] = loading + if output_retention is not UNSET: + field_dict["output_retention"] = output_retention + + return field_dict + + @classmethod + def from_dict(cls, src_dict: Mapping[str, Any]) -> Self: + d = dict(src_dict) + name = d.pop("name") + + tools = cast(list[str], d.pop("tools")) + + _loading = d.pop("loading", UNSET) + loading: ToolLoading | Unset + if isinstance(_loading, Unset): + loading = UNSET + else: + loading = ToolLoading._decode(_loading) + + output_retention = d.pop("output_retention", UNSET) + + tool_group = cls( + name=name, + tools=tools, + loading=loading, + output_retention=output_retention, + ) + + tool_group.additional_properties = d + return tool_group + + @property + def additional_keys(self) -> list[str]: + return list(self.additional_properties.keys()) + + def __getitem__(self, key: str) -> Any: + return self.additional_properties[key] + + def __setitem__(self, key: str, value: Any) -> None: + self.additional_properties[key] = value + + def __delitem__(self, key: str) -> None: + del self.additional_properties[key] + + def __contains__(self, key: str) -> bool: + return key in self.additional_properties diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_loading.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_loading.py new file mode 100644 index 0000000000..b89c7c5f9e --- /dev/null +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_loading.py @@ -0,0 +1,30 @@ +from enum import StrEnum + + +class ToolLoading(StrEnum): + DEFERRED = "deferred" + FULL = "full" + SUMMARY = "summary" + + def __str__(self) -> str: + return str(self.value) + + @classmethod + def _decode(cls, value: object) -> "ToolLoading": + """Reads the value from an answer. A string this client does not list yet is kept as a member of its own, + so an answer from a newer router still reads, and is written back as it came.""" + try: + return cls(value) + except ValueError: + if not isinstance(value, str): + raise + member = str.__new__(cls, value) + member._name_ = value + member._value_ = value + return member + + def __reduce_ex__(self, protocol: object) -> object: + """Pickles a member this client does not list through _decode, as Enum's own reduce would call cls(value).""" + if self._value_ in type(self)._value2member_map_: + return super().__reduce_ex__(protocol) + return type(self)._decode, (self._value_,) diff --git a/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_stats.py b/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_stats.py index 25833fdbfd..45e28d41a6 100644 --- a/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_stats.py +++ b/plugins/stream/vision_agents/plugins/stream/_generated/models/tool_stats.py @@ -21,6 +21,16 @@ class ToolStats: failed (int): Calls that ended in an error. p99_ms (float | None): tool (str): + avg_turns_before_use (float | None): Turns a session took before its first call, on average: 0 is a call on the + first turn. Null on the row over every tool. + definition_tokens (int | None): What the tool's whole definition, its name, description and schema, costs every + turn that carries it, estimated at four characters a token. Null when no call recorded it. + loading (str): How the latest call had the tool offered: full, summary or deferred. Empty when no call recorded + it, as calls made before it was recorded and built-in tools. + session_share (float | None): Sessions that called it, out of the window's sessions, from 0 to 1. Null when no + session took a turn. + sessions (int): Sessions that called it at least once. + summary_tokens (int | None): What its definition costs offered as a summary. Null unless its loading is summary. """ avg_ms: float | None @@ -29,6 +39,12 @@ class ToolStats: failed: int p99_ms: float | None tool: str + avg_turns_before_use: float | None + definition_tokens: int | None + loading: str + session_share: float | None + sessions: int + summary_tokens: int | None additional_properties: dict[str, Any] = _attrs_field(init=False, factory=dict) def to_dict(self) -> dict[str, Any]: @@ -46,6 +62,22 @@ def to_dict(self) -> dict[str, Any]: tool = self.tool + avg_turns_before_use: float | None + avg_turns_before_use = self.avg_turns_before_use + + definition_tokens: int | None + definition_tokens = self.definition_tokens + + loading = self.loading + + session_share: float | None + session_share = self.session_share + + sessions = self.sessions + + summary_tokens: int | None + summary_tokens = self.summary_tokens + field_dict: dict[str, Any] = {} field_dict.update(self.additional_properties) field_dict.update( @@ -56,6 +88,12 @@ def to_dict(self) -> dict[str, Any]: "failed": failed, "p99_ms": p99_ms, "tool": tool, + "avg_turns_before_use": avg_turns_before_use, + "definition_tokens": definition_tokens, + "loading": loading, + "session_share": session_share, + "sessions": sessions, + "summary_tokens": summary_tokens, } ) @@ -87,6 +125,40 @@ def _parse_p99_ms(data: object) -> float | None: tool = d.pop("tool") + def _parse_avg_turns_before_use(data: object) -> float | None: + if data is None: + return data + return cast(float | None, data) + + avg_turns_before_use = _parse_avg_turns_before_use( + d.pop("avg_turns_before_use") + ) + + def _parse_definition_tokens(data: object) -> int | None: + if data is None: + return data + return cast(int | None, data) + + definition_tokens = _parse_definition_tokens(d.pop("definition_tokens")) + + loading = d.pop("loading") + + def _parse_session_share(data: object) -> float | None: + if data is None: + return data + return cast(float | None, data) + + session_share = _parse_session_share(d.pop("session_share")) + + sessions = d.pop("sessions") + + def _parse_summary_tokens(data: object) -> int | None: + if data is None: + return data + return cast(int | None, data) + + summary_tokens = _parse_summary_tokens(d.pop("summary_tokens")) + tool_stats = cls( avg_ms=avg_ms, avg_tokens=avg_tokens, @@ -94,6 +166,12 @@ def _parse_p99_ms(data: object) -> float | None: failed=failed, p99_ms=p99_ms, tool=tool, + avg_turns_before_use=avg_turns_before_use, + definition_tokens=definition_tokens, + loading=loading, + session_share=session_share, + sessions=sessions, + summary_tokens=summary_tokens, ) tool_stats.additional_properties = d diff --git a/plugins/stream/vision_agents/plugins/stream/_tool_line.py b/plugins/stream/vision_agents/plugins/stream/_tool_line.py index 4e870aa56d..8a999910c2 100644 --- a/plugins/stream/vision_agents/plugins/stream/_tool_line.py +++ b/plugins/stream/vision_agents/plugins/stream/_tool_line.py @@ -54,16 +54,6 @@ def __init__(self, send: Send, holds: Holds) -> None: self._send = send self._holds = holds - async def update(self, call_id: str, text: str, verbatim: bool) -> None: - await self._send( - { - "type": "tool_update", - "tool_call_id": call_id, - "text": text, - "verbatim": verbatim, - } - ) - async def hold(self, call_id: str) -> bool: return await self._holds.ask( call_id, lambda: self._send({"type": "tool_hold", "tool_call_id": call_id}) diff --git a/plugins/stream/vision_agents/plugins/stream/client.py b/plugins/stream/vision_agents/plugins/stream/client.py index 9fa8cd3492..70606b44bd 100644 --- a/plugins/stream/vision_agents/plugins/stream/client.py +++ b/plugins/stream/vision_agents/plugins/stream/client.py @@ -24,6 +24,7 @@ GuestUserRequestCustom, ) from .account import OrganizationAccount, Usage +from .dispatch import Dispatch from .responses import RouterError, _deleted, _unwrapped from .router import Router from .sessions import Sessions @@ -130,12 +131,40 @@ def __init__( self.teams = Teams(self.backend, self.spend_limits) self.account = OrganizationAccount(self.backend) self.usage = Usage(self.backend) + # Registered once for every agent, run by `run_dispatch`, and offered only to an + # agent whose agent.yaml names them in a tool group. + self.tools = FunctionRegistry() @property def server_side(self) -> bool: """Whether this speaks for a process the customer runs rather than for a device.""" return self.backend.server_side + async def run_dispatch(self) -> None: + """Run the app's tools for every session they are offered to, until cancelled. + + Each agent is offered the ones a tool group of its agent.yaml names, wherever its + session was opened. A tool's own ``timeout`` is how long the router waits for one of + its calls; without one it waits 30 seconds. A worker that also answers calls or + messages hosts the tools with ``Dispatch.host(client)`` instead. + + Example: + ```python + client = stream.Client() + + + @client.tools.register(description="Look up the weather") + async def weather_lookup(location: str) -> str: + return await weather.today(location) + + + await client.run_dispatch() + ``` + """ + dispatch = Dispatch(backend=self.backend) + dispatch.host(self) + await dispatch.run() + def as_user(self, user: Union[str, dict[str, Any]], token: str) -> "Client": """A client acting for one end user, holding the token that proves it. @@ -321,51 +350,9 @@ class Agent: def __init__(self, backend: Backend, name: str): self.name = name - # The functions are held on the agent rather than per session, because a function - # registered once should not have to be registered again for the next conversation. - self.functions = FunctionRegistry() - self.sessions = Sessions(backend, name, self.functions) + self.sessions = Sessions(backend, name) self._backend = backend - def register( - self, - name: Optional[str] = None, - description: Optional[str] = None, - parameters_schema: Optional[dict[str, Any]] = None, - interruptible: bool = False, - timeout: float = 0.0, - needs_approval: str = "", - ): - """Register one of the caller's functions, which every conversation offers the model. - - The decorator the core registry provides, so the schema comes from the type hints and - the description from the docstring. A first parameter annotated `ToolCallContext` is - handed the call it runs for: - - @agent.register(needs_approval="Book flight {flight_id}?") - async def book_flight(tool_call_context: ToolCallContext, flight_id: str) -> str: - "Book a flight for the caller" - await tool_call_context.update("checking seats") - await tool_call_context.disallow_interruptions() - return await bookings.book(flight_id, key=tool_call_context.id) - - Args: - interruptible: Let the caller talking over the agent cancel a call in flight. - Off, the call keeps running and the model is told it is still going. - timeout: How long one call may take, in seconds. Zero takes the session's. - needs_approval: A question a person answers before each call runs, with - {name} for an argument. On a voice call the agent asks it out loud and - waits 15 seconds for a yes; in writing it goes to the app. - """ - return self.functions.register( - name=name, - description=description, - parameters_schema=parameters_schema, - interruptible=interruptible, - timeout=timeout, - needs_approval=needs_approval, - ) - async def config(self) -> Optional[AgentConfig]: """How the agent is configured, or None for a name nothing is stored under. diff --git a/plugins/stream/vision_agents/plugins/stream/config.py b/plugins/stream/vision_agents/plugins/stream/config.py index fe027353fb..da97eca068 100644 --- a/plugins/stream/vision_agents/plugins/stream/config.py +++ b/plugins/stream/vision_agents/plugins/stream/config.py @@ -26,7 +26,6 @@ AgentDispatch, AgentMode, AgentScheduling, - AgentTools, ChannelIdentity, ChannelLineRequest, ConnectionOwnerType, @@ -47,6 +46,8 @@ SessionVideo, SyncAgentRequest, SyncAgentRequestTags, + ToolGroup, + ToolLoading, SyncAgentResult, ) from ._generated.types import UNSET, Unset @@ -368,9 +369,15 @@ async def _declare_settings( await _connector_binding(client, entry) for entry in settings.connectors ] if settings.tools is not None: - body.tools = AgentTools() - if settings.tools.progressive is not None: - body.tools.progressive = settings.tools.progressive + body.tools = [ + ToolGroup( + name=group.name, + tools=list(group.tools), + loading=ToolLoading(group.loading) if group.loading else UNSET, + output_retention=group.output_retention or UNSET, + ) + for group in settings.tools + ] if settings.channels is not None: declared = AgentChannels() if settings.channels.whatsapp is not None: @@ -454,6 +461,8 @@ async def _connector_binding( require_approval=UNSET if entry.require_approval is None else list(entry.require_approval), + loading=ToolLoading(entry.loading) if entry.loading else UNSET, + output_retention=entry.output_retention or UNSET, ) diff --git a/plugins/stream/vision_agents/plugins/stream/dispatch.py b/plugins/stream/vision_agents/plugins/stream/dispatch.py index 621f1540c8..eba2f976a7 100644 --- a/plugins/stream/vision_agents/plugins/stream/dispatch.py +++ b/plugins/stream/vision_agents/plugins/stream/dispatch.py @@ -30,7 +30,7 @@ from .sessions import _tools if TYPE_CHECKING: - from . import client + from .client import Client logger = logging.getLogger(__name__) @@ -49,9 +49,8 @@ @dataclass class _Hosting: - """One set of functions this worker runs for every session under an agent id.""" + """One set of functions this worker runs for every agent of the app.""" - agent_id: str functions: FunctionRegistry tool_timeout: float @@ -98,6 +97,7 @@ def __init__( customer_id: Optional[str] = None, capacity: int = 4, report_every: float = 15.0, + backend: Optional[Backend] = None, ): """Wait for one customer's calls on a router. @@ -110,6 +110,8 @@ def __init__( process can actually answer. report_every: How often to tell the router how this process is doing, in seconds. + backend: A backend to use as it is, such as a client's, instead of building one + from `url` and `customer_id`. Raises: ValueError: If capacity is not a number of calls. @@ -117,7 +119,7 @@ def __init__( if capacity < 1: raise ValueError("a worker that can hold no calls cannot answer any") - self.backend = Backend(url=url, customer_id=customer_id) + self.backend = backend or Backend(url=url, customer_id=customer_id) self.capacity = capacity self.report_every = report_every @@ -200,35 +202,34 @@ def register(handler: MessageHandler) -> MessageHandler: return register - def host(self, agent: "client.Agent", tool_timeout: float = 0.0) -> None: - """Run an agent's functions for every session opened under it, whoever opened it. + def host(self, client: "Client", tool_timeout: float = 0.0) -> None: + """Run the app's tools for every session that is offered them, whoever opened it. - A session's own functions run in the process that opened it, which is no use to a - conversation opened from a browser. Hosting is the other direction: the router offers - these functions to each session naming the agent and sends every call to a worker - hosting them. Call before `run`. + The tools are registered once on the client, for every agent: the router offers one + to a session only where a tool group of its agent's agent.yaml names it, and sends + every call of it to a worker hosting it. `Client.run_dispatch` does this alone, for + a worker with no calls or messages to answer. Call before `run`. Example: ```python - agent = stream.Client().agent("stream-support") + client = stream.Client() - @agent.register(description="Read the SDK's source") + @client.tools.register(description="Read the SDK's source") async def investigate_sdk(sdk: str) -> str: return await read_source(sdk) - dispatch.host(agent, tool_timeout=60) + dispatch.host(client, tool_timeout=60) ``` Args: - agent: The agent whose sessions are offered its functions, hosted under its - name. + client: The client whose tools every agent may be offered. tool_timeout: How long the router waits for one tool call to be answered, in seconds, before telling the model it failed. Not how long the worker runs. Zero takes the router's default of 30 seconds. """ - self._hosted.append(_Hosting(agent.name, agent.functions, tool_timeout)) + self._hosted.append(_Hosting(client.tools, tool_timeout)) async def get_or_create_agent( self, message: InboundMessage, create_agent: AgentFactory @@ -428,14 +429,13 @@ async def _read(self, socket: Socket) -> None: cancel(call_id, self._holds, self._hosted_calls.get(key)) elif kind == "hosting": logger.info( - "the router sends the tools of %s here", frame.get("agent_id", "") + "the router sends the calls of %s here", frame.get("tools", []) ) elif kind == "hosting_refused": # Not worth reconnecting: a worker whose tools were refused is one nobody # will call, and saying so beats sitting connected looking healthy. raise RouterError( - f"the router refused to host tools for agent " - f"{frame.get('agent_id', '')}: {frame.get('reason', '')}" + f"the router refused to host tools: {frame.get('reason', '')}" ) elif kind == "pong": self._latency_ms = ( @@ -451,7 +451,7 @@ async def _host(self) -> None: await self._tell( { "type": "host_tools", - "agent_id": offer.agent_id, + "every_agent": True, "tools": [tool.to_dict() for tool in _tools(offer.functions)], "timeout_ms": int(offer.tool_timeout * 1000), } diff --git a/plugins/stream/vision_agents/plugins/stream/folder.py b/plugins/stream/vision_agents/plugins/stream/folder.py index 5c007ad0f3..393716b736 100644 --- a/plugins/stream/vision_agents/plugins/stream/folder.py +++ b/plugins/stream/vision_agents/plugins/stream/folder.py @@ -168,15 +168,27 @@ class ConnectorSettings: require_approval: list[str] | None = None """Names or patterns of the tools whose every call waits for the session's user to allow it. None sends no require_approval key.""" + loading: str = "" + """How the connector's tools are offered, as a tool group's. Empty is full.""" + output_retention: str = "" + """How long sessions keep what the connector's tools were given and gave back: none, or + a duration such as 30d. Empty is as long as the session.""" + + +TOOL_LOADINGS = ("full", "summary", "deferred") @dataclass -class ToolSettings: - """How connector tools are offered.""" +class ToolGroupSettings: + """One group of the agent's functions, and how they are offered.""" - progressive: bool | None = None - """Offer each tool by a summary, the first call to it returning its full description - instead of running it. None when the file says nothing about it.""" + name: str + tools: list[str] + loading: str = "" + """full, summary or deferred. Empty is full.""" + output_retention: str = "" + """How long sessions keep what the group's tools were given and gave back: none, or a + duration such as 30d. Empty is as long as the session.""" @dataclass @@ -244,8 +256,10 @@ class Settings: connectors: list[ConnectorSettings] | None = None """Connectors the agent binds. None when the file says nothing about them, which leaves whatever is stored; an empty list removes them.""" - tools: ToolSettings | None = None - """How connector tools are offered. None when the file says nothing about it.""" + tools: list[ToolGroupSettings] | None = None + """The groups naming the functions this process provides that the agent may offer: a + function no group names is never offered. None when the file says nothing, which + leaves the groups stored.""" channels: ChannelsSettings | None = None """Lines the agent answers on besides Stream Chat. The provider's credentials live on the router, connected once for the app; this only names the numbers.""" @@ -636,6 +650,8 @@ def _connectors(path: Path, value: object) -> list[ConnectorSettings] | None: "required", "sandbox", "require_approval", + "loading", + "output_retention", } if extra: raise ValueError(f"{path} unknown connectors setting: {sorted(extra)[0]}") @@ -676,6 +692,8 @@ def _connectors(path: Path, value: object) -> list[ConnectorSettings] | None: required=required, sandbox=bool(sandbox), require_approval=_connector_require_approval(path, item), + loading=_loading(path, f'connector "{name}"', item.get("loading")), + output_retention=_text(item.get("output_retention")), ) ) return named @@ -728,18 +746,55 @@ def _greeting(path: Path, value: object) -> GreetingSettings | None: return GreetingSettings(text=text, mode=_word(value.get("mode"))) -def _tools(path: Path, value: object) -> ToolSettings | None: +def _tools(path: Path, value: object) -> list[ToolGroupSettings] | None: if value is None: return None - if not isinstance(value, dict): - raise ValueError(f"{path} should give tools as a mapping") - extra = set(value) - {"progressive"} - if extra: - raise ValueError(f"{path} unknown tools setting: {sorted(extra)[0]}") - progressive = value.get("progressive") - if progressive is not None and not isinstance(progressive, bool): - raise ValueError(f"{path} should give tools progressive as true or false") - return ToolSettings(progressive=progressive) + if isinstance(value, dict): + raise ValueError( + f"{path}: tools is a list of groups, each with a name and its tools, such as " + "- {name: booking, loading: full, tools: [lookup_booking]}; progressive: true " + "is now loading: summary, and output_retention moves onto each group and connector" + ) + if not isinstance(value, list): + raise ValueError(f"{path} should give tools as a list of groups") + groups: list[ToolGroupSettings] = [] + for item in value: + if not isinstance(item, dict): + raise ValueError( + f"{path}: a tool group is a mapping with a name and its tools" + ) + extra = set(item) - {"name", "loading", "tools", "output_retention"} + if extra: + raise ValueError(f"{path} unknown tool group setting: {sorted(extra)[0]}") + name = _text(item.get("name")) + if not name: + raise ValueError(f"{path}: a tool group needs a name") + tools = item.get("tools") + if not isinstance(tools, list) or not tools: + raise ValueError(f'{path}: tool group "{name}" names no tools') + if not all(isinstance(tool, str) for tool in tools): + raise ValueError( + f'{path}: tool group "{name}" should name its tools as text' + ) + groups.append( + ToolGroupSettings( + name=name, + tools=list(tools), + loading=_loading(path, f'tool group "{name}"', item.get("loading")), + output_retention=_text(item.get("output_retention")), + ) + ) + return groups + + +def _loading(path: Path, owner: str, value: object) -> str: + """A tool group's or connector's loading, empty when it gives none.""" + loading = _text(value) + if loading and loading not in TOOL_LOADINGS: + raise ValueError( + f"{path}: {owner}: loading is {', '.join(TOOL_LOADINGS)}, not {loading!r}" + ) + return loading def _channels(path: Path, value: object) -> ChannelsSettings | None: diff --git a/plugins/stream/vision_agents/plugins/stream/sessions.py b/plugins/stream/vision_agents/plugins/stream/sessions.py index 125d8c2f4c..478b72be70 100644 --- a/plugins/stream/vision_agents/plugins/stream/sessions.py +++ b/plugins/stream/vision_agents/plugins/stream/sessions.py @@ -153,6 +153,9 @@ class SessionOptions: interim: Also report what the caller is part way through saying. decisions: Report the router's own routing decisions. Off here by default, because it is the router explaining itself several times a second. + tools: The caller's own functions for this conversation only, run in this process. + The app's tools (``Client.tools``) reach a conversation through + ``Client.run_dispatch`` instead. """ id: str = "" @@ -172,6 +175,7 @@ class SessionOptions: connector_bindings: Optional[dict[str, str]] = None return_url: str = "" timezone: str = "" + tools: Optional[FunctionRegistry] = None interim: bool = False decisions: bool = False @@ -223,10 +227,9 @@ def to_dict(self) -> dict[str, Any]: class Sessions: """One agent's conversations: the ones being held and the ones that were.""" - def __init__(self, backend: Backend, agent: str, functions: FunctionRegistry): + def __init__(self, backend: Backend, agent: str): self._backend = backend self._agent = agent - self._functions = functions async def create(self, options: Optional[SessionOptions] = None) -> "Session": """Open a conversation and start watching it. @@ -242,7 +245,9 @@ async def create(self, options: Optional[SessionOptions] = None) -> "Session": ), f"opening a session with {self._agent}", ) - return await Session.watching(self._backend, row, self._functions, options) + return await Session.watching( + self._backend, row, options.tools or FunctionRegistry(), options + ) async def resume( self, id: str, options: Optional[SessionOptions] = None @@ -250,8 +255,9 @@ async def resume( """Carry on a conversation held in writing, by the id of the session it was held in, and start watching it. One that ended is reopened with what was said in it.""" row = await self.get(id) + options = options or SessionOptions() return await Session.watching( - self._backend, row, self._functions, options or SessionOptions() + self._backend, row, options.tools or FunctionRegistry(), options ) async def query( @@ -488,7 +494,7 @@ def _request(self, options: SessionOptions) -> CreateSessionRequest: if options.incognito: request.incognito = True - declared = _tools(self._functions) + declared = _tools(options.tools or FunctionRegistry()) if declared: request.tools = declared return request @@ -888,7 +894,7 @@ def _stream(self): def _sessions(self) -> Sessions: """The resource methods that act on a session by id, which never need its agent.""" - return Sessions(self._backend, "", self._functions) + return Sessions(self._backend, "") async def _command(self, frame: dict[str, Any]) -> None: """Act on the session over the socket it is being watched on.""" diff --git a/sdks/dart/stream_agents/lib/src/core/agent_event.dart b/sdks/dart/stream_agents/lib/src/core/agent_event.dart index 7164e7f9b9..6880eebb49 100644 --- a/sdks/dart/stream_agents/lib/src/core/agent_event.dart +++ b/sdks/dart/stream_agents/lib/src/core/agent_event.dart @@ -48,8 +48,13 @@ enum AgentEventKind { taskCancelled('task_cancelled'), toolCall('tool_call'), toolCancel('tool_cancel'), + toolHeld('tool_held'), toolStarted('tool_started'), + toolUpdated('tool_updated'), toolRan('tool_ran'), + toolCallStarted('tool_call.started'), + toolCallProgress('tool_call.progress'), + toolCallFinished('tool_call.finished'), transferred('transferred'), pressed('pressed'), lookedUp('looked_up'), @@ -136,9 +141,35 @@ final class AgentEvent { arguments: _text(fields['arguments']), requestId: _text(fields['request_id']), turnId: turnId, + approved: fields['approved'] == true, ) : null; + /// The call's whole state on a `tool_call.started`, `tool_call.progress` or + /// `tool_call.finished`, or null for any other event. + ToolCallState? get toolCallState => switch (kind) { + AgentEventKind.toolCallStarted || + AgentEventKind.toolCallProgress || + AgentEventKind.toolCallFinished when _text(fields['tool_call_id']).isNotEmpty => ToolCallState( + id: _text(fields['tool_call_id']), + tool: _text(fields['tool']), + turnId: turnId, + status: ToolCallStatus._byWire[_text(fields['status'])] ?? ToolCallStatus.unknown, + progress: _text(fields['progress']), + startedAt: parseRouterDate(_text(fields['started_at'])), + duration: Duration( + milliseconds: switch (fields['duration_ms']) { + final num ms => ms.toInt(), + _ => 0, + }, + ), + approvedAt: parseRouterDate(_text(fields['approved_at'])), + rejectedAt: parseRouterDate(_text(fields['rejected_at'])), + error: _text(fields['error']), + ), + _ => null, + }; + /// A connector tool call the provider refused because the person's own connection lacks /// access it asked for, and the step-up consent the router began for it, or null when this /// event is not one. The old grant keeps working until the step-up succeeds, and the same @@ -200,11 +231,15 @@ final class ToolCall { required this.arguments, this.requestId = '', this.turnId = '', + this.approved = false, }); final String id; final String name; + /// Whether the person already said yes to it, on a voice call, so nobody asks again. + final bool approved; + /// The arguments as the model wrote them, which is a JSON object encoded as a string. final String arguments; @@ -224,4 +259,95 @@ final class ToolCall { } } +/// Where a tool call stands. [unknown] is a status the router added after this SDK shipped. +enum ToolCallStatus { + inProgress('in_progress'), + + /// Held for a person's approval. + waiting('waiting'), + success('success'), + failed('failed'), + cancelled('cancelled'), + unknown(''); + + const ToolCallStatus(this.wire); + + /// The router's name for it. + final String wire; + + /// Whether the call is over. + bool get isFinished => this == success || this == failed || this == cancelled; + + static final Map _byWire = { + for (final status in values) status.wire: status, + }; +} + +/// One tool call as the session last reported it, whichever process runs it. +final class ToolCallState { + const ToolCallState({ + required this.id, + required this.tool, + required this.status, + this.turnId = '', + this.progress = '', + this.startedAt, + this.duration = Duration.zero, + this.approvedAt, + this.rejectedAt, + this.error = '', + }); + + /// The call's `tool_call_id`. + final String id; + final String tool; + final String turnId; + final ToolCallStatus status; + + /// The last thing said about the call while it runs, such as "Searching for flights", or the + /// question it waits on. + final String progress; + final DateTime? startedAt; + + /// From [startedAt] to this report; once finished, how long the call took. + final Duration duration; + + /// When a person allowed the call. + final DateTime? approvedAt; + + /// When a person declined the call, or its wait for them passed. + final DateTime? rejectedAt; + + /// Why a failed call failed, or what a declined one shows. + final String error; + + @override + bool operator ==(Object other) => + other is ToolCallState && + other.id == id && + other.tool == tool && + other.turnId == turnId && + other.status == status && + other.progress == progress && + other.startedAt == startedAt && + other.duration == duration && + other.approvedAt == approvedAt && + other.rejectedAt == rejectedAt && + other.error == error; + + @override + int get hashCode => Object.hash( + id, + tool, + turnId, + status, + progress, + startedAt, + duration, + approvedAt, + rejectedAt, + error, + ); +} + String _text(Object? value) => value is String ? value : ''; diff --git a/sdks/dart/stream_agents/lib/src/core/agent_session.dart b/sdks/dart/stream_agents/lib/src/core/agent_session.dart index 9c5b7f235f..b7d0a4073e 100644 --- a/sdks/dart/stream_agents/lib/src/core/agent_session.dart +++ b/sdks/dart/stream_agents/lib/src/core/agent_session.dart @@ -119,6 +119,7 @@ final class AgentSession { final _conversation = LiveValueController(const Conversation()); final _connection = LiveValueController(Connection.notStarted); final _approvals = LiveValueController>(const []); + final _toolCalls = LiveValueController>(const {}); final Set<_Subscriber> _subscribers = {}; final Set _running = {}; final Set _cancelled = {}; @@ -142,6 +143,10 @@ final class AgentSession { /// first. LiveValue> get approvals => _approvals; + /// Every tool call of the session by its id, as it was last reported: running, waiting for a + /// person's approval, or how it ended, whichever process runs it. Finished calls stay. + LiveValue> get toolCalls => _toolCalls; + List get turns => _conversation.value.turns; ConversationState get state => _conversation.value.state; bool get isConnected => _connection.value is Connected; @@ -215,6 +220,11 @@ final class AgentSession { }); } + /// Calls [handler] with every event of [kind], such as [AgentEventKind.toolCallFinished], + /// until the subscription is cancelled or the conversation ends. + StreamSubscription on(AgentEventKind kind, void Function(AgentEvent event) handler) => + events().where((event) => event.kind == kind).listen(handler); + /// Speaks this without going through the model. /// /// [interrupt] stops what the agent is saying first, rather than waiting for it to finish. @@ -502,6 +512,9 @@ final class AgentSession { void _apply(AgentEvent event) { _conversation.value = _conversation.value.apply(event); + if (event.toolCallState case final call?) { + _toolCalls.value = {..._toolCalls.value, call.id: call}; + } switch (event.kind) { case AgentEventKind.toolCall: _answer(event.toolCall!, run: event['run'] != false); @@ -541,13 +554,16 @@ final class AgentSession { } return; } + if (!run) { + return; + } // Session events go to observers as well as to whoever owns the tool. A worker elsewhere // may own this one, and an observer must not answer it first. final tool = _tools[call.name]; if (tool == null) { return; } - if (tool.approval case final approval?) { + if (tool.approval case final approval? when !call.approved) { _approvals.value = [..._approvals.value, ToolApprovalRequest._(call, approval, tool)]; } else { _run(call, tool); @@ -665,6 +681,7 @@ final class AgentSession { _conversation.close(); _connection.close(); _approvals.close(); + _toolCalls.close(); } } diff --git a/sdks/dart/stream_agents/test/core/session_test.dart b/sdks/dart/stream_agents/test/core/session_test.dart index 26893abc41..f298157901 100644 --- a/sdks/dart/stream_agents/test/core/session_test.dart +++ b/sdks/dart/stream_agents/test/core/session_test.dart @@ -462,6 +462,99 @@ void main() { await chat.close(); }); + test( + 'a call meant for another watcher is not run, and one approved by voice is not asked again', + () async { + final ran = []; + final chat = await agents.chat( + SessionOptions( + tools: [ + AgentTool( + name: 'lookup_order', + description: 'Look up an order.', + run: (_) async => 'found', + ), + AgentTool( + name: 'issue_refund', + description: 'Refund an order.', + approval: const ToolApproval(title: 'Refund it?'), + run: (_) async { + ran.add('refund'); + return 'refunded'; + }, + ), + ], + ), + ); + final socket = await router.socket(); + + socket + ..send( + '{"type":"tool_call","id":"elsewhere","name":"lookup_order","arguments":"{}","run":false}', + ) + ..send( + '{"type":"tool_call","id":"approved","name":"issue_refund","arguments":"{}","approved":true}', + ) + ..send('{"type":"tool_call","id":"here","name":"lookup_order","arguments":"{}"}'); + + final answered = { + (await socket.next())['tool_call_id'], + (await socket.next())['tool_call_id'], + }; + expect(answered, {'approved', 'here'}); + expect(ran, ['refund']); + expect(chat.approvals.value, isEmpty); + await chat.close(); + }, + ); + + test( + 'keeps each tool call\'s last state, and a handler hears its kind until cancelled', + () async { + final chat = await agents.chat(); + final socket = await router.socket(); + final progress = []; + final finished = []; + late StreamSubscription first; + first = chat.on(AgentEventKind.toolCallProgress, (event) { + progress.add(event['progress'] as String); + first.cancel(); + }); + chat.on( + AgentEventKind.toolCallFinished, + (event) => finished.add(event['tool_call_id'] as String), + ); + const call = + '"tool_call_id":"c1","tool":"search_flights","turn_id":"t1","started_at":"2026-11-18T12:00:00Z"'; + + socket + ..send('{"type":"tool_call.started",$call,"status":"waiting","duration_ms":0}') + ..send( + '{"type":"tool_call.progress",$call,"status":"in_progress","duration_ms":100,' + '"progress":"Searching for flights","approved_at":"2026-11-18T12:00:00.1Z"}', + ) + ..send( + '{"type":"tool_call.progress",$call,"status":"in_progress","duration_ms":200,' + '"progress":"Comparing prices","approved_at":"2026-11-18T12:00:00.1Z"}', + ) + ..send( + '{"type":"tool_call.finished",$call,"status":"success","duration_ms":300,' + '"approved_at":"2026-11-18T12:00:00.1Z"}', + ); + await until(() => chat.toolCalls.value['c1']?.status.isFinished ?? false); + await until(() => finished.isNotEmpty); + + final last = chat.toolCalls.value['c1']!; + expect(last.status, ToolCallStatus.success); + expect(last.duration, const Duration(milliseconds: 300)); + expect(last.progress, '', reason: 'the finished frame carries the whole state'); + expect(last.approvedAt, DateTime.utc(2026, 11, 18, 12, 0, 0, 100)); + expect(progress, ['Searching for flights']); + expect(finished, ['c1']); + await chat.close(); + }, + ); + test('a tool call the router cancelled is not answered', () async { final release = Completer(); final chat = await agents.chat( diff --git a/sdks/dotnet/README.md b/sdks/dotnet/README.md index f378c2c3c5..8af4490f55 100644 --- a/sdks/dotnet/README.md +++ b/sdks/dotnet/README.md @@ -80,7 +80,6 @@ for as its first argument: agent.Tools.Register("book_flight", "Book a flight for the caller", async (toolCallContext, booking, cancellationToken) => { - await toolCallContext.UpdateAsync("checking seats", cancellationToken); await toolCallContext.DisallowInterruptionsAsync(cancellationToken); return await BookAsync(booking.FlightId, toolCallContext.User.Id, idempotencyKey: toolCallContext.Id); }, @@ -98,9 +97,7 @@ agent.Tools.Register("book_flight", "Book a flight for the call `toolCallContext.Id` is the id the model gave the call, which a call sent again keeps, so it is the key to make a side effect happen once; `SessionId`, `TurnId`, `User` and `Conversation` say -where it came from. `UpdateAsync(text)` says how the call is going: a voice agent tells the -caller in its own words once nobody is talking. `SayAsync(text)` has it say `text` as written, -which a speech-to-speech model may paraphrase. `DisallowInterruptionsAsync` keeps the call going +where it came from. `DisallowInterruptionsAsync` keeps the call going from then on, for a write that must not stop halfway, and throws `CallNotHeldException` when the router had already cancelled it. A cancelled call has its token cancelled and is not answered. A call the router addresses to another watcher (`run: false`) is left alone. @@ -178,21 +175,21 @@ showing it the photos sent with it (`Images`, by URL), and is cancelled, after the work in hand has finished. A connection the router drops is opened again, telling the router how much is still being handled. -A worker can also host an agent's tools for every session opened under that agent, including -sessions opened from a browser. The router sends each call here. Handlers are optional: +A worker can also host the app's tools, registered once on the client for every agent, for any +session, including sessions opened from a browser. The router offers a tool to a session only +where a tool group of its agent's agent.yaml names it, and sends each call here: ```csharp using var client = new VisionAgentsClient(); -var agent = client.Agent("my-agent"); -agent.Tools.Register("get_weather", "Get the current weather for a city", +client.Tools.Register("get_weather", "Get the current weather for a city", (weather, cancellationToken) => WeatherAsync(weather.City, cancellationToken)); -await new Dispatch(new DispatchOptions { Client = client }).Host(agent, toolTimeout: TimeSpan.FromMinutes(1)).RunAsync(stopping); +await client.RunDispatchAsync(stopping); ``` A hosted tool takes the same `interruptible`, `timeout` and `approval` (asked on a voice call), can take its call, and is cancelled when the router gives up on it. If the router refuses the -tools, `RunAsync` throws `InvalidOperationException` with the -reason. +tools, `RunDispatchAsync` throws `InvalidOperationException` with the reason. A worker that also +answers calls or messages hosts them with `dispatch.Host(client)` before `RunAsync`. ## An agent written down as a directory diff --git a/sdks/dotnet/src/GetStream.VisionAgents/Dispatch.cs b/sdks/dotnet/src/GetStream.VisionAgents/Dispatch.cs index 1e4556daf0..33af1d7def 100644 --- a/sdks/dotnet/src/GetStream.VisionAgents/Dispatch.cs +++ b/sdks/dotnet/src/GetStream.VisionAgents/Dispatch.cs @@ -41,8 +41,8 @@ public sealed record DispatchOptions /// shared between them. /// /// -/// A worker can also an agent's tools: the router offers them to every -/// session opened under that agent, and sends each call down this connection. +/// A worker can also the app's tools: the router offers them to every +/// session whose agent names them in a tool group, and sends each call down this connection. /// /// public sealed class Dispatch @@ -57,7 +57,7 @@ public sealed class Dispatch private static readonly TimeSpan SteadyAfter = TimeSpan.FromMinutes(1); private readonly TimeSpan _reportEvery; - private readonly List<(string AgentId, Tools Tools, TimeSpan Timeout)> _hosted = []; + private readonly List<(Tools Tools, TimeSpan Timeout)> _hosted = []; private readonly ConcurrentDictionary _running = new(); private readonly ConcurrentDictionary _hostedCalls = new(); private readonly Holds _holds = new(); @@ -142,22 +142,22 @@ public Dispatch WaitForMessage(Func handler) } /// - /// Runs an agent's for every session opened under it, whoever opened it. + /// Runs the app's for every session they are offered to, whoever opened it. /// /// - /// A session's own tools run in the process that opened it, which is no use to a - /// conversation opened from a browser. Hosting is the other direction: the router offers - /// them to each session whose agent id or config name is the agent's - /// , and sends every call here. Call before . + /// The tools are registered once on the client, for every agent: the router offers one to + /// a session only where a tool group of its agent's agent.yaml names it, and sends every + /// call here. does this alone, for a + /// worker with no calls or messages to answer. Call before . /// - /// The agent, usually , whose tools are hosted. + /// The client whose tools are hosted. /// /// How long the router waits for one tool call to be answered before telling the model it /// failed, not how long the worker runs; null takes the router's default of 30 seconds. /// - public Dispatch Host(Agent agent, TimeSpan? toolTimeout = null) + public Dispatch Host(VisionAgentsClient client, TimeSpan? toolTimeout = null) { - _hosted.Add((agent.Name, agent.Tools, toolTimeout ?? TimeSpan.Zero)); + _hosted.Add((client.Tools, toolTimeout ?? TimeSpan.Zero)); return this; } @@ -381,9 +381,9 @@ await HandleAsync(frame.Text("work_id"), _onMessage is { } onMessage ? () => onM break; case "ready": WorkerId = frame.Text("worker_id"); - foreach (var (agentId, tools, timeout) in _hosted) + foreach (var (tools, timeout) in _hosted) { - await TellAsync(Frames.Of("host_tools", ("agent_id", agentId), ("tools", tools.Declared()), + await TellAsync(Frames.Of("host_tools", ("every_agent", true), ("tools", tools.Declared()), ("timeout_ms", (long)timeout.TotalMilliseconds))).ConfigureAwait(false); } break; @@ -403,13 +403,12 @@ await TellAsync(Frames.Of("host_tools", ("agent_id", agentId), ("tools", tools.D _holds.Answer(key, false); break; case "hosting": - Trace.TraceInformation("the router sends tools for agent {0} to this worker", frame.Text("agent_id")); + Trace.TraceInformation("the router sends the app's tools to this worker"); break; case "hosting_refused": // A worker whose tools were refused is one nobody will call; saying so beats // sitting connected looking healthy. - throw new InvalidOperationException( - $"the router refused to host tools for agent {frame.Text("agent_id")}: {frame.Text("reason")}"); + throw new InvalidOperationException($"the router refused to host tools: {frame.Text("reason")}"); case "pong": _latencyMs = (_clock.Elapsed.TotalSeconds - frame.Number("at")) * 1000; _pong.TrySetResult(); diff --git a/sdks/dotnet/src/GetStream.VisionAgents/ToolCallContext.cs b/sdks/dotnet/src/GetStream.VisionAgents/ToolCallContext.cs index 8f0b23d9cc..c2c81c37fc 100644 --- a/sdks/dotnet/src/GetStream.VisionAgents/ToolCallContext.cs +++ b/sdks/dotnet/src/GetStream.VisionAgents/ToolCallContext.cs @@ -52,17 +52,6 @@ public sealed class ToolCallContext /// Whether a person allowed this call before it ran. public bool Approved { get; init; } - /// - /// Tells the people in the conversation how the call is going, such as "Found three - /// flights, comparing prices". The agent puts it in its own words. - /// - public Task UpdateAsync(string text, CancellationToken cancellationToken = default) => - SendUpdateAsync(text, false, cancellationToken); - - /// , said word for word. - public Task SayAsync(string text, CancellationToken cancellationToken = default) => - SendUpdateAsync(text, true, cancellationToken); - /// /// Keeps the call going if the person talks over the agent from here on, for a call that /// is past the point of taking back, such as one that has started a payment. @@ -86,11 +75,6 @@ public async Task DisallowInterruptionsAsync(CancellationToken cancellationToken } } - private Task SendUpdateAsync(string text, bool verbatim, CancellationToken cancellationToken) => - _line is null - ? Task.CompletedTask - : _line.Send(Frames.Of("tool_update", ("tool_call_id", Id), ("text", text), ("verbatim", verbatim)), cancellationToken); - /// /// The call a tool_call frame asks for. stands in for /// a frame that does not name its session. diff --git a/sdks/dotnet/src/GetStream.VisionAgents/VisionAgentsClient.cs b/sdks/dotnet/src/GetStream.VisionAgents/VisionAgentsClient.cs index aaaae5941d..99179772e7 100644 --- a/sdks/dotnet/src/GetStream.VisionAgents/VisionAgentsClient.cs +++ b/sdks/dotnet/src/GetStream.VisionAgents/VisionAgentsClient.cs @@ -92,11 +92,37 @@ private VisionAgentsClient(HttpClient httpClient, Backend backend, bool owns) internal HttpClient Http => _http; + /// + /// The app's tools, registered once for every agent and run by . + /// An agent is offered one only where a tool group of its agent.yaml names it. + /// + public Tools Tools { get; private init; } = new(); + /// /// An agent addressed by the config it is stored under. No request is made. /// public Agent Agent(string config) => new(new AgentOptions { Config = config, Client = this }); + /// + /// Runs the app's for every session they are offered to, whoever opened + /// it, until is cancelled. + /// + /// + /// A tool's own timeout is how long the router waits for one of its calls; without one it + /// waits 30 seconds. A worker that also answers calls or messages hosts the tools with + /// instead. + /// + /// + /// + /// var client = new VisionAgentsClient(); + /// client.Tools.Register<Weather, string>("get_weather", "Get the current weather for a city", + /// (weather, cancellationToken) => WeatherAsync(weather.City, cancellationToken)); + /// await client.RunDispatchAsync(); + /// + /// + public Task RunDispatchAsync(CancellationToken cancellationToken = default) => + new Dispatch(new DispatchOptions { Client = this }).Host(this).RunAsync(cancellationToken); + /// /// A client acting for one end user, holding the token that proves it. /// @@ -115,14 +141,14 @@ public VisionAgentsClient AsUser(string userId, string token) { throw new ConfigurationException($"there is no token for {userId} to hold"); } - return new VisionAgentsClient(_http, Backend.As(userId, token), owns: false); + return new VisionAgentsClient(_http, Backend.As(userId, token), owns: false) { Tools = Tools }; } /// /// This client's own credential, speaking for an end user: their conversations are /// reached, and what is written there is still the server's. /// - internal VisionAgentsClient ActingFor(string userId) => new(_http, Backend.Acting(userId), owns: false); + internal VisionAgentsClient ActingFor(string userId) => new(_http, Backend.Acting(userId), owns: false) { Tools = Tools }; /// A client acting for a guest, which is what their conversations belong to. public VisionAgentsClient AsGuest(GuestUser guest) => AsUser(guest.Id, guest.Token); diff --git a/sdks/dotnet/tests/GetStream.VisionAgents.Tests/AgentTests.cs b/sdks/dotnet/tests/GetStream.VisionAgents.Tests/AgentTests.cs index 598e1d4731..8bd9060fc9 100644 --- a/sdks/dotnet/tests/GetStream.VisionAgents.Tests/AgentTests.cs +++ b/sdks/dotnet/tests/GetStream.VisionAgents.Tests/AgentTests.cs @@ -330,6 +330,34 @@ public async Task ASessionIsSteeredOverItsSocket() Assert.Empty(router.To("POST", "/v1/agents/sessions/s1/stop")); } + [Fact] + public async Task ACancelledTokenStopsTheCallerWaitingForAStalledSendButNotTheFrame() + { + await using var router = await TestRouter.StartAsync(); + router.On("POST", "/v1/agents/sessions", 201, Fixtures.Session()); + var thrown = new TaskCompletionSource(); + var said = new TaskCompletionSource(); + router.OnSocket(Events, async peer => + { + peer.Pause(); + await thrown.Task.WaitAsync(TestRouter.Patience); + peer.Resume(); + said.SetResult(await peer.ReceiveAsync("say")); + }); + using var client = Fixtures.Client(router); + await using var agent = new Agent(new AgentOptions { Name = "jean", Client = client }); + var cancel = TestContext.Current.CancellationToken; + var session = await agent.ChatAsync(cancellationToken: cancel); + + using var caller = new CancellationTokenSource(TimeSpan.FromMilliseconds(300)); + await Assert.ThrowsAnyAsync( + () => session.SayAsync(new string('x', 32 << 20), caller.Token).WaitAsync(TestRouter.Patience, cancel)); + thrown.SetResult(); + + Assert.Equal(32 << 20, (await said.Task.WaitAsync(TestRouter.Patience, cancel)).Text("text").Length); + await session.CloseAsync(cancel); + } + [Fact] public async Task AResponseIsCreatedInTheConversationTheAgentOpened() { diff --git a/sdks/dotnet/tests/GetStream.VisionAgents.Tests/DispatchTests.cs b/sdks/dotnet/tests/GetStream.VisionAgents.Tests/DispatchTests.cs index abed016d28..9840a1ba36 100644 --- a/sdks/dotnet/tests/GetStream.VisionAgents.Tests/DispatchTests.cs +++ b/sdks/dotnet/tests/GetStream.VisionAgents.Tests/DispatchTests.cs @@ -425,22 +425,23 @@ public async Task AHostedToolIsDeclaredAndAnsweredOverTheDispatchSocket() { await peer.SendAsync(new { type = "ready", worker_id = "worker-1" }); declared = await peer.ReceiveAsync("host_tools"); - await peer.SendAsync(new { type = "hosting", agent_id = "stream-support" }); + await peer.SendAsync(new { type = "hosting", every_agent = true }); await peer.SendAsync(new { type = "tool_call", id = "call-1", session_id = "s", name = "investigate_sdk", arguments = """{"sdk":"android"}""" }); answered.Add(await peer.ReceiveAsync("tool_result")); await peer.SendAsync(new { type = "tool_call", id = "call-2", session_id = "s", name = "delete_everything", arguments = "{}" }); answered.Add(await peer.ReceiveAsync("tool_result")); }); using var client = Fixtures.Client(router); - var agent = client.Agent("stream-support"); - agent.Tools.Register("investigate_sdk", "Read SDK source", + client.Tools.Register("investigate_sdk", "Read SDK source", (investigation, _) => Task.FromResult("read " + investigation.Sdk)); - var dispatch = new Dispatch(new DispatchOptions { Client = client }).Host(agent, TimeSpan.FromMinutes(1)); + var dispatch = new Dispatch(new DispatchOptions { Client = client }).Host(client, TimeSpan.FromMinutes(1)); await dispatch.RunAsync(TestContext.Current.CancellationToken); Assert.Equal("", router.Only("GET", "/v1/dispatch").Query["handles"]); - Assert.Equal(("stream-support", 60000), (declared.Text("agent_id"), declared!["timeout_ms"]!.GetValue())); + Assert.True(declared!["every_agent"]!.GetValue()); + Assert.False(declared.ContainsKey("agent_id")); + Assert.Equal(60000, declared["timeout_ms"]!.GetValue()); var tool = Assert.Single(declared["tools"]!.AsArray())!; Assert.Equal(("investigate_sdk", "Read SDK source"), (tool["name"]!.GetValue(), tool["description"]!.GetValue())); Assert.NotNull(tool["parameters"]!["properties"]!["sdk"]); @@ -449,6 +450,31 @@ public async Task AHostedToolIsDeclaredAndAnsweredOverTheDispatchSocket() Assert.Equal(0, dispatch.Active); } + [Fact] + public async Task RunDispatchHostsTheClientsToolsForEveryAgent() + { + await using var router = await TestRouter.StartAsync(); + JsonObject? declared = null; + JsonObject? answered = null; + router.OnSocket("/v1/dispatch", async peer => + { + await peer.SendAsync(new { type = "ready", worker_id = "worker-1" }); + declared = await peer.ReceiveAsync("host_tools"); + await peer.SendAsync(new { type = "tool_call", id = "call-1", session_id = "s", name = "investigate_sdk", arguments = """{"sdk":"ios"}""" }); + answered = await peer.ReceiveAsync("tool_result"); + }); + using var client = Fixtures.Client(router); + client.Tools.Register("investigate_sdk", "Read SDK source", + (investigation, _) => Task.FromResult("read " + investigation.Sdk)); + + await client.RunDispatchAsync(TestContext.Current.CancellationToken); + + Assert.Equal("", router.Only("GET", "/v1/dispatch").Query["handles"]); + Assert.True(declared!["every_agent"]!.GetValue()); + Assert.Equal("investigate_sdk", Assert.Single(declared["tools"]!.AsArray())!["name"]!.GetValue()); + Assert.Equal(("call-1", "read ios"), (answered.Text("id"), answered.Text("output"))); + } + [Fact] public async Task AWorkerTheRouterDropsReconnectsAndHostsAgain() { @@ -479,10 +505,9 @@ public async Task AWorkerTheRouterDropsReconnectsAndHostsAgain() } }); using var client = Fixtures.Client(router); - var agent = client.Agent("stream-support"); - agent.Tools.Register("investigate_sdk", "Read SDK source", (_, _) => Task.FromResult("")); + client.Tools.Register("investigate_sdk", "Read SDK source", (_, _) => Task.FromResult("")); var dispatch = new Dispatch(new DispatchOptions { Client = client }) { FirstRetry = TimeSpan.FromMilliseconds(10) } - .Host(agent); + .Host(client); using var stop = CancellationTokenSource.CreateLinkedTokenSource(TestContext.Current.CancellationToken); var running = dispatch.RunAsync(stop.Token); @@ -491,7 +516,7 @@ public async Task AWorkerTheRouterDropsReconnectsAndHostsAgain() await running.WaitAsync(TestRouter.Patience, TestContext.Current.CancellationToken); Assert.Equal("worker-3", dispatch.WorkerId); - Assert.Equal(["stream-support", "stream-support", "stream-support"], hosted.Select(frame => frame.Text("agent_id"))); + Assert.Equal([true, true, true], hosted.Select(frame => frame["every_agent"]!.GetValue())); Assert.Equal(0, hosted.First()["timeout_ms"]!.GetValue()); } @@ -503,16 +528,15 @@ public async Task AWorkerWhoseToolsAreRefusedStopsWaiting() { await peer.SendAsync(new { type = "ready", worker_id = "worker-1" }); await peer.ReceiveAsync("host_tools"); - await peer.SendAsync(new { type = "hosting_refused", agent_id = "stream-support", reason = "hosting no tools is not hosting" }); + await peer.SendAsync(new { type = "hosting_refused", reason = "hosting no tools is not hosting" }); await peer.Closed; }); using var client = Fixtures.Client(router); - var dispatch = new Dispatch(new DispatchOptions { Client = client }).Host(client.Agent("stream-support")); + var dispatch = new Dispatch(new DispatchOptions { Client = client }).Host(client); var refused = await Assert.ThrowsAsync( () => dispatch.RunAsync(TestContext.Current.CancellationToken).WaitAsync(TestRouter.Patience, TestContext.Current.CancellationToken)); - Assert.Contains("stream-support", refused.Message); Assert.Contains("hosting no tools is not hosting", refused.Message); } diff --git a/sdks/dotnet/tests/GetStream.VisionAgents.Tests/ToolCallTests.cs b/sdks/dotnet/tests/GetStream.VisionAgents.Tests/ToolCallTests.cs index a9960c313f..d32778f28b 100644 --- a/sdks/dotnet/tests/GetStream.VisionAgents.Tests/ToolCallTests.cs +++ b/sdks/dotnet/tests/GetStream.VisionAgents.Tests/ToolCallTests.cs @@ -28,26 +28,20 @@ await peer.SendAsync(new user = new { id = "u1", name = "Ada", custom = new { tier = "gold" } }, conversation = new { cid = "messaging:c1", custom = new { } }, }); - sent.Add(await peer.ReceiveAsync()); - sent.Add(await peer.ReceiveAsync()); sent.Add(await peer.ReceiveAsync("tool_result")); await Finish(peer); }); await using var agent = Agent(router); - agent.Tools.Register("book", "Books a flight", [], async (ToolCallContext toolCallContext, JsonElement _, CancellationToken cancellationToken) => + agent.Tools.Register("book", "Books a flight", [], (ToolCallContext toolCallContext, JsonElement _, CancellationToken _) => { - await toolCallContext.UpdateAsync("Looking at flights", cancellationToken); - await toolCallContext.SayAsync("One moment.", cancellationToken); var user = toolCallContext.User; - return (object?)$"{toolCallContext.Id} {toolCallContext.SessionId} {toolCallContext.TurnId} {user.Id} {user.Name} " + - $"{user.Custom.Text("tier")} {toolCallContext.Conversation.Cid} {toolCallContext.Approved}"; + return Task.FromResult($"{toolCallContext.Id} {toolCallContext.SessionId} {toolCallContext.TurnId} {user.Id} {user.Name} " + + $"{user.Custom.Text("tier")} {toolCallContext.Conversation.Cid} {toolCallContext.Approved}"); }); await Converse(agent); - Assert.Equal("""{"type":"tool_update","tool_call_id":"t1","text":"Looking at flights","verbatim":false}""", sent[0].ToJsonString()); - Assert.Equal("""{"type":"tool_update","tool_call_id":"t1","text":"One moment.","verbatim":true}""", sent[1].ToJsonString()); - Assert.Equal("t1 s1 turn-1 u1 Ada gold messaging:c1 True", sent[2].Text("output")); + Assert.Equal("t1 s1 turn-1 u1 Ada gold messaging:c1 True", sent[0].Text("output")); } [Fact] @@ -135,132 +129,6 @@ public async Task AHoldAskedForTooLateIsCancelledWithTheCall() Assert.Equal("f1", Assert.Single(answered).Text("tool_call_id")); } - [Fact] - public async Task ACallCancelledWhileItIsSendingDoesNotEndTheSession() - { - await using var router = await TestRouter.StartAsync(); - router.On("POST", "/v1/agents/sessions", 201, Fixtures.Session()); - var sending = new TaskCompletionSource(); - var cancelled = new TaskCompletionSource(); - JsonObject? answered = null; - router.OnSocket(Events, async peer => - { - // Nothing reads the update, so it is still being written when the cancel arrives. - peer.Pause(); - await peer.SendAsync(new { type = "tool_call", id = "t1", name = "talk", arguments = "{}" }); - await sending.Task.WaitAsync(TestRouter.Patience); - await peer.SendAsync(new { type = "tool_cancel", id = "t1" }); - await cancelled.Task.WaitAsync(TestRouter.Patience); - peer.Resume(); - await peer.SendAsync(new { type = "tool_call", id = "f1", name = "fast", arguments = "{}" }); - answered = await peer.ReceiveAsync("tool_result"); - await Finish(peer); - }); - await using var agent = Agent(router); - agent.Tools - .Register("talk", "Talks at length", [], async (ToolCallContext toolCallContext, JsonElement _, CancellationToken cancellationToken) => - { - var update = toolCallContext.UpdateAsync(new string('x', 32 << 20), cancellationToken); - cancellationToken.Register(cancelled.SetResult); - sending.SetResult(); - await update; - cancellationToken.ThrowIfCancellationRequested(); - return (object?)"done"; - }) - .Register("fast", "Does not", [], (JsonElement _, CancellationToken _) => Task.FromResult("done")); - - await Converse(agent); - - Assert.Equal("f1", answered.Text("tool_call_id")); - } - - [Fact] - public async Task ACancelledTokenStopsTheCallerWaitingForAStalledSendButNotTheFrame() - { - await using var router = await TestRouter.StartAsync(); - router.On("POST", "/v1/agents/sessions", 201, Fixtures.Session()); - var thrown = new TaskCompletionSource(); - JsonObject? update = null; - JsonObject? answered = null; - router.OnSocket(Events, async peer => - { - peer.Pause(); - await peer.SendAsync(new { type = "tool_call", id = "t1", name = "talk", arguments = "{}" }); - await thrown.Task.WaitAsync(TestRouter.Patience); - peer.Resume(); - update = await peer.ReceiveAsync("tool_update"); - answered = await peer.ReceiveAsync("tool_result"); - await Finish(peer); - }); - await using var agent = Agent(router); - agent.Tools - .Register("talk", "Talks at length", [], async (ToolCallContext toolCallContext, JsonElement _, CancellationToken _) => - { - using var caller = new CancellationTokenSource(TimeSpan.FromMilliseconds(300)); - try - { - await toolCallContext.UpdateAsync(new string('x', 32 << 20), caller.Token).WaitAsync(TestRouter.Patience); - } - catch (OperationCanceledException) - { - thrown.SetResult(); - } - return (object?)"done"; - }); - - await Converse(agent); - - Assert.Equal(32 << 20, update.Text("text").Length); - Assert.Equal("t1", answered.Text("tool_call_id")); - } - - [Fact] - public async Task ACancelledCallSendsNothingMore() - { - await using var router = await TestRouter.StartAsync(); - router.On("POST", "/v1/agents/sessions", 201, Fixtures.Session()); - var attempted = new TaskCompletionSource(); - JsonObject? next = null; - router.OnSocket(Events, async peer => - { - await peer.SendAsync(new { type = "tool_call", id = "t1", name = "talk", arguments = "{}" }); - await peer.SendAsync(new { type = "tool_cancel", id = "t1" }); - await attempted.Task.WaitAsync(TestRouter.Patience); - await peer.SendAsync(new { type = "tool_call", id = "f1", name = "fast", arguments = "{}" }); - next = await peer.ReceiveAsync(); - await Finish(peer); - }); - await using var agent = Agent(router); - agent.Tools - .Register("talk", "Talks", [], async (ToolCallContext toolCallContext, JsonElement _, CancellationToken cancellationToken) => - { - try - { - await Task.Delay(Timeout.Infinite, cancellationToken); - } - catch (OperationCanceledException) - { - // Cancelled by the router; now try to talk anyway. - } - try - { - await toolCallContext.UpdateAsync("late", cancellationToken); - } - catch (OperationCanceledException) - { - attempted.SetResult(); - throw; - } - attempted.SetResult(); - return (object?)"x"; - }) - .Register("fast", "Does not", [], (JsonElement _, CancellationToken _) => Task.FromResult("done")); - - await Converse(agent); - - Assert.Equal(("tool_result", "f1"), (next.Text("type"), next.Text("tool_call_id"))); - } - [Fact] public async Task ACallThatOutlastsItsTimeoutFails() { @@ -356,10 +224,9 @@ public async Task AHostedToolIsDeclaredWithItsPoliciesAndAnsweredByCallId() answered = await peer.ReceiveAsync("tool_result"); }); using var client = Fixtures.Client(router); - var agent = client.Agent("payments"); - Payments(agent.Tools); + Payments(client.Tools); - await new Dispatch(new DispatchOptions { Client = client }).Host(agent, TimeSpan.FromMinutes(1)) + await new Dispatch(new DispatchOptions { Client = client }).Host(client, TimeSpan.FromMinutes(1)) .RunAsync(TestContext.Current.CancellationToken); var tools = declared!["tools"]!.AsArray().ToDictionary(tool => tool.Text("name"), tool => tool!.AsObject()); @@ -392,10 +259,9 @@ public async Task AHostedCallCancelledByAnInterruptionIsNotAnswered() } }); using var client = Fixtures.Client(router); - var agent = client.Agent("payments"); - Payments(agent.Tools); + Payments(client.Tools); - await new Dispatch(new DispatchOptions { Client = client }).Host(agent, TimeSpan.FromMinutes(1)) + await new Dispatch(new DispatchOptions { Client = client }).Host(client, TimeSpan.FromMinutes(1)) .RunAsync(TestContext.Current.CancellationToken); Assert.Equal("c2", Assert.Single(answered, frame => frame.Text("type") == "tool_result").Text("id")); @@ -417,10 +283,9 @@ public async Task AHostedCallHeldThroughInterruptionsIsAnswered() sent.Add(await peer.ReceiveAsync("tool_result")); }); using var client = Fixtures.Client(router); - var agent = client.Agent("payments"); - Payments(agent.Tools); + Payments(client.Tools); - await new Dispatch(new DispatchOptions { Client = client }).Host(agent, TimeSpan.FromMinutes(1)) + await new Dispatch(new DispatchOptions { Client = client }).Host(client, TimeSpan.FromMinutes(1)) .RunAsync(TestContext.Current.CancellationToken); Assert.Equal("""{"type":"tool_hold","tool_call_id":"c1"}""", sent[0].ToJsonString()); diff --git a/sdks/go/README.md b/sdks/go/README.md index cacf693f9a..729b63b7a2 100644 --- a/sdks/go/README.md +++ b/sdks/go/README.md @@ -183,13 +183,11 @@ run, whose `ID` is the same when the router sends a call again, so a function th something can key on it. A call sent again while it is still running joins it, and one that finished is answered with what it returned, without running twice. -A call that takes a while can say how it is going, and protect a write from the caller -talking over the agent: +A call can protect a write from the caller talking over the agent: ```go func(ctx context.Context, args RebookFlight) (any, error) { toolCallContext := tools.ToolCallContextFrom(ctx) - toolCallContext.Update(ctx, "Checking seats on the new flight") // Read only: the caller interrupting cancels ctx, and nothing has changed. seat, err := airline.HoldSeat(ctx, args.NewFlightID) @@ -199,17 +197,13 @@ func(ctx context.Context, args RebookFlight) (any, error) { // From here on, the caller interrupting no longer cancels the toolCallContext. ctx = toolCallContext.DisallowInterruptions(ctx) - toolCallContext.Say(ctx, "Got a seat, moving your booking now.") return airline.Rebook(ctx, args.BookingID, seat, toolCallContext.ID) } ``` -`toolCallContext.Update` hands the agent a status, which it tells a voice caller in its own -words once nobody is talking, and which a conversation shows on the call's step. -`toolCallContext.Say` has it say the words as written, or post them in text; a speech-to-speech -model may paraphrase them. `DisallowInterruptions` returns a context only the call's timeout and -the socket ending cancel. When the call was already stopped, that context comes back cancelled, -so the write never starts. Outside a call, all three do nothing. +`DisallowInterruptions` returns a context only the call's timeout and the socket ending cancel. +When the call was already stopped, that context comes back cancelled, so the write never +starts. Outside a call, it does nothing. Struct tags describe arguments beyond `schema`: @@ -478,17 +472,18 @@ agents.NewDispatch(agents.DispatchOptions{ }) ``` -A worker can also host an agent's tools for every session opened under it, including one a +Tools are registered once for the app, on the client, and each agent is offered the ones a +tool group of its `agent.yaml` names. A worker runs them for every session, including one a browser opened, where the session's own process cannot reach what the tool needs: ```go -agent := api.Agent("my-agent") -agent.Tools().Add(LookupOrder{}) -dispatch.Host(agent, time.Minute) +api.Tools().Add(LookupOrder{}) +err := api.RunDispatch(ctx) ``` -The router offers them to each session naming the agent and sends every call here. `Run` -works with only hosted tools and no handler, and stops if the router refuses them. +The router sends every call of them here. A worker that also answers calls or messages hosts +them with `dispatch.Host(api, time.Minute)` before `Run`. Either stops if the router refuses +the tools. `sdks/go/examples/dispatch` is the whole of it, and `examples/voice_agents/chat_support` is the Python one. diff --git a/sdks/go/acceleration/generated.go b/sdks/go/acceleration/generated.go index 03b3a7b107..51d6ea794e 100644 --- a/sdks/go/acceleration/generated.go +++ b/sdks/go/acceleration/generated.go @@ -3040,6 +3040,75 @@ func (e ToolApprovalCommandType) Valid() bool { } } +// Defines values for ToolCallFrameType. +const ( + ToolCallFinished ToolCallFrameType = "tool_call.finished" + ToolCallProgress ToolCallFrameType = "tool_call.progress" + ToolCallStarted ToolCallFrameType = "tool_call.started" +) + +// Valid indicates whether the value is a known member of the ToolCallFrameType enum. +func (e ToolCallFrameType) Valid() bool { + switch e { + case ToolCallFinished: + return true + case ToolCallProgress: + return true + case ToolCallStarted: + return true + default: + return false + } +} + +// Defines values for ToolCallStatus. +const ( + ToolCallStatusCancelled ToolCallStatus = "cancelled" + ToolCallStatusFailed ToolCallStatus = "failed" + ToolCallStatusInProgress ToolCallStatus = "in_progress" + ToolCallStatusSuccess ToolCallStatus = "success" + ToolCallStatusWaiting ToolCallStatus = "waiting" +) + +// Valid indicates whether the value is a known member of the ToolCallStatus enum. +func (e ToolCallStatus) Valid() bool { + switch e { + case ToolCallStatusCancelled: + return true + case ToolCallStatusFailed: + return true + case ToolCallStatusInProgress: + return true + case ToolCallStatusSuccess: + return true + case ToolCallStatusWaiting: + return true + default: + return false + } +} + +// Defines values for ToolLoading. +const ( + ToolLoadingDeferred ToolLoading = "deferred" + ToolLoadingFull ToolLoading = "full" + ToolLoadingSummary ToolLoading = "summary" +) + +// Valid indicates whether the value is a known member of the ToolLoading enum. +func (e ToolLoading) Valid() bool { + switch e { + case ToolLoadingDeferred: + return true + case ToolLoadingFull: + return true + case ToolLoadingSummary: + return true + default: + return false + } +} + // Defines values for ToolResultCommandType. const ( ToolResultCommandTypeToolResult ToolResultCommandType = "tool_result" @@ -3709,9 +3778,9 @@ type AgentConfig struct { SyncHash *string `json:"sync_hash,omitempty"` Tags *map[string]string `json:"tags,omitempty"` - // Tools How an agent is offered its tools, and how long what they were given and gave back is kept. - Tools *AgentTools `json:"tools,omitempty"` - Tts *string `json:"tts,omitempty"` + // Tools The functions an SDK provides that the agent may offer, in named groups. Empty when there are none, and then the agent offers no function. + Tools *[]ToolGroup `json:"tools,omitempty"` + Tts *string `json:"tts,omitempty"` // UiInstructions Absent when the agent is taught no UI. UiInstructions *string `json:"ui_instructions,omitempty"` @@ -3817,9 +3886,9 @@ type AgentConfigPatch struct { Subagent *string `json:"subagent,omitempty"` Tags *map[string]string `json:"tags,omitempty"` - // Tools How an agent is offered its tools, and how long what they were given and gave back is kept. - Tools *AgentTools `json:"tools,omitempty"` - Tts *string `json:"tts,omitempty"` + // Tools Replaces the tool groups whole; an empty list removes them all. + Tools *[]ToolGroup `json:"tools,omitempty"` + Tts *string `json:"tts,omitempty"` // UiInstructions What teaches the agent's model to answer with UI. An empty string removes them. UiInstructions *string `json:"ui_instructions,omitempty"` @@ -3953,9 +4022,9 @@ type AgentConfigRequest struct { // Tags Cost labels, carried onto every request a session using it makes. The team tag is also the team those sessions are counted under, its spend limits included, when whoever opens one names none: neither the caller's token nor the session's team field. It is 1 to 59 letters, digits, _ . and -, and a config naming another team tag is refused. A config tagged draft_of, naming the config it copies, is a test copy: it answers no message on a channel connection and subscribes to no event, which stay with the live config, and it may bind a channel connection another config binds. Tags *map[string]string `json:"tags,omitempty"` - // Tools How an agent is offered its tools, and how long what they were given and gave back is kept. - Tools *AgentTools `json:"tools,omitempty"` - Tts *string `json:"tts,omitempty"` + // Tools The functions an SDK provides that the agent may offer, in named groups, each with how it is loaded. A function an SDK registers, declares on a session or hosts in a worker is offered only when a group names it: a session declaring one that none names is refused (400 tool_not_configured), and a worker's is left out. The router's own tools, connector tools and other agents are not named here. Omitted or null on an update, the groups stored stay; an empty list removes them all. The mapping this field once was, with progressive and output_retention, is refused: loading summary and output_retention on a group or a connector binding replaced them. + Tools *[]ToolGroup `json:"tools,omitempty"` + Tts *string `json:"tts,omitempty"` // UiInstructions What teaches the agent's model to answer with UI, such as the section the SDK compiles from agent.yaml's ui key. Every session is told them after the instructions, unless it was opened with ui false. Empty teaches none. Left out on an update, the stored ones stay. UiInstructions *string `json:"ui_instructions,omitempty"` @@ -3990,9 +4059,17 @@ type AgentConnectorBinding struct { // Events MCP events the binding's fixed connection is subscribed to, each opening a text conversation from the config when it arrives. Subscribed when the connection is next validated. Only a fixed binding may declare events: a session binding's connection is picked when a session opens, and an event arrives with no session open. Events *[]ConnectorBindingEvent `json:"events,omitempty"` + // Loading How a tool is offered to the model. full sends its whole description and input schema from the start. summary sends the first line of its description and its arguments without their descriptions, and the first call returns the full description and schema instead of running it: one more model turn for each tool a conversation uses, for less context on every turn. deferred offers nothing until the model needs it: an agent with a deferred tool is offered search_tool, which finds deferred tools by what they do, and load_tool, which returns their full descriptions and offers them from the model's next step on. + Loading *ToolLoading `json:"loading,omitempty"` + // Name The alias, unique within the config: a lowercase letter, then up to 62 lowercase letters, digits, - or _, never __ and not ending in _. The model is offered each tool as __, split back at the first __, so a __ inside the alias or a _ at its end would split it in the wrong place. Name string `json:"name"` + // OutputRetention How long the agent's sessions keep what the binding's tools were given and gave back on the turns they record, as a tool group's output_retention. Omitted or empty, it is kept as long as the session. + // + // Example: 30d + OutputRetention *string `json:"output_retention,omitempty"` + // Policy How a binding's tool calls behave around speech and interruptions. Every field is optional, and a field left out keeps today's behaviour. Policy *ConnectorBindingPolicy `json:"policy,omitempty"` @@ -4155,28 +4232,23 @@ type AgentScheduling struct { // AgentStats defines model for AgentStats. type AgentStats struct { + // AvgTurns Turns a session took in the window, on average. A turn is a reply to the person, not a rewound one or a delegated task's. Null when no session took one. + AvgTurns *float64 `json:"avg_turns"` + // PeakAt When that peak was first reached. Null when no session was open in the window. PeakAt *time.Time `json:"peak_at"` // PeakConcurrentSessions The most sessions open at the same time, each one an agent running. A session still open counts up to now. - PeakConcurrentSessions int64 `json:"peak_concurrent_sessions"` - ToolCalls ToolStats `json:"tool_calls"` + PeakConcurrentSessions int64 `json:"peak_concurrent_sessions"` + + // Sessions Sessions that took a turn in the window: what a tool's session_share is out of. + Sessions int64 `json:"sessions"` + ToolCalls ToolStats `json:"tool_calls"` // Tools One row per tool, most called first. Tools []ToolStats `json:"tools"` } -// AgentTools How an agent is offered its tools, and how long what they were given and gave back is kept. -type AgentTools struct { - // OutputRetention How long this agent's sessions keep what any of its tools were given and gave back on the turns they record: a tool's arguments, its result, a search's query and the wording of a tool's error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed. Left out on a create, or empty, it is kept as long as the session; left out on an update, the stored setting stays, and an empty string removes it. The app's and the organization's policies (tool_output_retention) can shorten it and never lengthen it, and the shortest applies. It applies to tool calls recorded after it is set. - // - // Example: 30d - OutputRetention *string `json:"output_retention,omitempty"` - - // Progressive Offer each tool by the first line of its description, with its arguments' descriptions left out, and have the first call to a tool return its full description and input schema instead of running it. It saves context on an agent with many tools, at the cost of one more model turn for each tool a conversation uses. Off by default. Left out on an update, the stored setting stays. - Progressive *bool `json:"progressive,omitempty"` -} - // AgentUser defines model for AgentUser. type AgentUser struct { // IncludePhoneHistory true when the user reads all of what the number ever said. Left out otherwise. @@ -6590,7 +6662,7 @@ type EqualsOrIn1 struct { // ErrorDetail defines model for ErrorDetail. type ErrorDetail struct { - // Code What went wrong, for a program to branch on. Every type has a code of its own name (invalid_request, unauthenticated, forbidden, payment_required, not_found, method_not_allowed, not_acceptable, conflict, gone, payload_too_large, unsupported_media_type, rate_limited, internal_error, unavailable, timeout) that a failure has when nothing names it better. The others are validation_failed, missing_customer, missing_organization, server_side_only, not_configured (this deployment does not offer the feature), modality_not_routed, unsynced_changes (a sync asked to check would write over somebody's edits), name_taken (a 409: another agent config, router config or voice, or another skill of the same agent config, already has the name, so another name will do), channel_connection_taken (a 409: another live agent config binds the channel connection as fixed, and one agent answers a channel connection's messages, so the message names that config), call_unanswered (a 409: the call placed for a phone session ended with nobody answering, and the message names how: busy, no_answer, failed or canceled. Opening the session is refused, or, when the status came while the agent was joining the call, the session is ended at once and kept as ended), no_oauth_client (a 400: a consent found no OAuth client for the connector, and the message names each way the connector lets the app give it one), invalid_config (a 400: an agent config a run names does not start a session), insufficient_credits (a 402: the organization's credit is spent, so work that costs money is refused until credit is added; a model that costs nothing is still served), system_limit_reached (a 402: the organization's spend hard limit for the day or the month is reached, whatever its credit, and the message names the interval, the amount and when it resets), spend_limit_reached (a 402: one of the organization's own spend limits is spent, on the organization, an app, a key, a usage category or a team, and the message says which and when it resets, so retrying before then is refused again), user_limit_reached (a 429: a spend limit on the one end user the request is for is spent, which stops that user and no one else, and Retry-After says when its window ends), spend_check_unavailable (a 503: the router could not share what the organization is spending with the other routers for a while, so work that costs money is refused until it can; a model that costs nothing is still served, and Retry-After says to try again in a few seconds), budget_moved (a 400: a policy sent a budget, which spend limits replaced), limit_above_parent (a 400: a spend limit is above a limit of its parent in the same interval, and the message names the parent's limit), limit_exists (a 409: the subject already has a limit at that level and interval, and the message names its id), no_limit (a 404: the subject has no spend limit to reset or grant past), use_team (a 400: a category starting team: was sent, where the team field names a team), invalid_expiry (a 400: a grant's expires_at is not in the future, or is more than 366 days away), too_many_grants (a 409: the organization has 20 live grants, as many as it may), tier_limit_reached (the organization's tier limits refuse the work, and limit says which: a 403 for a call or a transfer, to a country the tier does not reach, past its minutes or recipients of the day, or past its calls at once; a 400 for an agent config whose sandbox is bigger or more open than the organization's sandbox limits allow. The message names the tier and the limit), account_suspended (a 403: the organization is suspended, and every request is refused, one for a model that costs nothing included, until it is restored; the message says to contact support), concurrency_limited (a 429: the app has as many runs in flight as it may, and Retry-After says when to try again), model_failed (a 503: the model failed or gave no answer), prompt_injection (a 400: the prompt injection screen refused the input of a run or a tool result it read, which retrying an unchanged input does not fix), image_unavailable (a 400: an image a run was handed could not be fetched or is not a JPEG, PNG, WebP or GIF, which retrying the same image does not fix), document_too_large (a 413: a file sent with a response has more pages or text than one may, alone or with the files the conversation already holds, and the message names the limit), document_unreadable (a 400: a file sent with a response could not be read, such as one with a password, one that is damaged or one with no text in it, which retrying the same file does not fix), invalid_usage_categories (a 400: the usage categories the request names, in the token's usage_categories and team claims or in X-Stream-Usage-Categories, are more than 10, one is not 1 to 64 characters of A-Z, a-z, 0-9, _ . : -, more than one names a team, or the team is over 59 characters), memory_unavailable (a 503: the memory store could not delete what was asked, so nothing that waits on it was deleted either, and the same request later can succeed), billing_not_enabled (a 404: this deployment bills nobody, or the caller's app is in no organization it bills, so there is no account to read), provider_takes_no_key (a 400: a key was put for a provider whose work is never routed on a customer's own key here, and the message names the ones that are), campaign_locked (a 409: the campaign's state does not allow the change, such as a launched campaign's sequence, or resuming one that is running), campaign_not_ready (a 400: a launch found the campaign missing what it needs to ring anybody, and the message says what), consent_required (a 400: a launch did not say the people on the campaign agreed to be called), branch_limit_reached (a 409: the conversation has as many edit and regenerate points as Stream Chat keeps for it, 100, so a regenerate or an edit is refused and the conversation is continued in a new chat; asking and reading are unaffected), branch_too_long (a 409: the branch an activation would bring back is more than 100 messages to send again, so it is continued in a new chat), branch_unavailable (a 410: the copies of a hidden branch were deleted, as a memory truncate deletes them, so it can be neither read nor brought back), switch_unfinished (a 409: a regenerate, an edit or an activation of the session failed after Stream Chat may have rewound, and finishing it still fails because Stream Chat does not answer, so the session's questions, regenerates, edits and activations are refused meanwhile; try again: each one tries to finish it first, so it clears once Stream Chat answers), and _not_found for agent_config, call, campaign, channel_account, command, connection, grant, knowledge_document, knowledge_url, limit, provider_key, router_config, scheduled_task, session, simulation, simulation_run, skill and voice. A scheduled task also has invalid_schedule (a schedule that does not read: at and in both or neither, a date, time or duration that does not parse, or until without repeat), schedule_in_past, schedule_too_far (a one-shot, or an in, more than a year ahead), invalid_timezone, timezone_unknown (no zone was given and neither the user nor the config has one), invalid_repeat, interval_too_short (a run moved to less than 15 minutes from now and earlier than the run it replaces) and too_many_tasks (a 409: the user is at their active-task limit). More may be added, so a client should expect one it does not know. + // Code What went wrong, for a program to branch on. Every type has a code of its own name (invalid_request, unauthenticated, forbidden, payment_required, not_found, method_not_allowed, not_acceptable, conflict, gone, payload_too_large, unsupported_media_type, rate_limited, internal_error, unavailable, timeout) that a failure has when nothing names it better. The others are validation_failed, missing_customer, missing_organization, server_side_only, not_configured (this deployment does not offer the feature), modality_not_routed, unsynced_changes (a sync asked to check would write over somebody's edits), name_taken (a 409: another agent config, router config or voice, or another skill of the same agent config, already has the name, so another name will do), channel_connection_taken (a 409: another live agent config binds the channel connection as fixed, and one agent answers a channel connection's messages, so the message names that config), call_unanswered (a 409: the call placed for a phone session ended with nobody answering, and the message names how: busy, no_answer, failed or canceled. Opening the session is refused, or, when the status came while the agent was joining the call, the session is ended at once and kept as ended), no_oauth_client (a 400: a consent found no OAuth client for the connector, and the message names each way the connector lets the app give it one), invalid_config (a 400: an agent config a run names does not start a session), tool_not_configured (a 400: a session under an agent config was declared or given a function that none of the config's tool groups names, and the message names it), insufficient_credits (a 402: the organization's credit is spent, so work that costs money is refused until credit is added; a model that costs nothing is still served), system_limit_reached (a 402: the organization's spend hard limit for the day or the month is reached, whatever its credit, and the message names the interval, the amount and when it resets), spend_limit_reached (a 402: one of the organization's own spend limits is spent, on the organization, an app, a key, a usage category or a team, and the message says which and when it resets, so retrying before then is refused again), user_limit_reached (a 429: a spend limit on the one end user the request is for is spent, which stops that user and no one else, and Retry-After says when its window ends), spend_check_unavailable (a 503: the router could not share what the organization is spending with the other routers for a while, so work that costs money is refused until it can; a model that costs nothing is still served, and Retry-After says to try again in a few seconds), budget_moved (a 400: a policy sent a budget, which spend limits replaced), limit_above_parent (a 400: a spend limit is above a limit of its parent in the same interval, and the message names the parent's limit), limit_exists (a 409: the subject already has a limit at that level and interval, and the message names its id), no_limit (a 404: the subject has no spend limit to reset or grant past), use_team (a 400: a category starting team: was sent, where the team field names a team), invalid_expiry (a 400: a grant's expires_at is not in the future, or is more than 366 days away), too_many_grants (a 409: the organization has 20 live grants, as many as it may), tier_limit_reached (the organization's tier limits refuse the work, and limit says which: a 403 for a call or a transfer, to a country the tier does not reach, past its minutes or recipients of the day, or past its calls at once; a 400 for an agent config whose sandbox is bigger or more open than the organization's sandbox limits allow. The message names the tier and the limit), account_suspended (a 403: the organization is suspended, and every request is refused, one for a model that costs nothing included, until it is restored; the message says to contact support), concurrency_limited (a 429: the app has as many runs in flight as it may, and Retry-After says when to try again), model_failed (a 503: the model failed or gave no answer), prompt_injection (a 400: the prompt injection screen refused the input of a run or a tool result it read, which retrying an unchanged input does not fix), image_unavailable (a 400: an image a run was handed could not be fetched or is not a JPEG, PNG, WebP or GIF, which retrying the same image does not fix), document_too_large (a 413: a file sent with a response has more pages or text than one may, alone or with the files the conversation already holds, and the message names the limit), document_unreadable (a 400: a file sent with a response could not be read, such as one with a password, one that is damaged or one with no text in it, which retrying the same file does not fix), invalid_usage_categories (a 400: the usage categories the request names, in the token's usage_categories and team claims or in X-Stream-Usage-Categories, are more than 10, one is not 1 to 64 characters of A-Z, a-z, 0-9, _ . : -, more than one names a team, or the team is over 59 characters), memory_unavailable (a 503: the memory store could not delete what was asked, so nothing that waits on it was deleted either, and the same request later can succeed), billing_not_enabled (a 404: this deployment bills nobody, or the caller's app is in no organization it bills, so there is no account to read), provider_takes_no_key (a 400: a key was put for a provider whose work is never routed on a customer's own key here, and the message names the ones that are), campaign_locked (a 409: the campaign's state does not allow the change, such as a launched campaign's sequence, or resuming one that is running), campaign_not_ready (a 400: a launch found the campaign missing what it needs to ring anybody, and the message says what), consent_required (a 400: a launch did not say the people on the campaign agreed to be called), branch_limit_reached (a 409: the conversation has as many edit and regenerate points as Stream Chat keeps for it, 100, so a regenerate or an edit is refused and the conversation is continued in a new chat; asking and reading are unaffected), branch_too_long (a 409: the branch an activation would bring back is more than 100 messages to send again, so it is continued in a new chat), branch_unavailable (a 410: the copies of a hidden branch were deleted, as a memory truncate deletes them, so it can be neither read nor brought back), switch_unfinished (a 409: a regenerate, an edit or an activation of the session failed after Stream Chat may have rewound, and finishing it still fails because Stream Chat does not answer, so the session's questions, regenerates, edits and activations are refused meanwhile; try again: each one tries to finish it first, so it clears once Stream Chat answers), and _not_found for agent_config, call, campaign, channel_account, command, connection, grant, knowledge_document, knowledge_url, limit, provider_key, router_config, scheduled_task, session, simulation, simulation_run, skill and voice. A scheduled task also has invalid_schedule (a schedule that does not read: at and in both or neither, a date, time or duration that does not parse, or until without repeat), schedule_in_past, schedule_too_far (a one-shot, or an in, more than a year ahead), invalid_timezone, timezone_unknown (no zone was given and neither the user nor the config has one), invalid_repeat, interval_too_short (a run moved to less than 15 minutes from now and earlier than the run it replaces) and too_many_tasks (a 409: the user is at their active-task limit). More may be added, so a client should expect one it does not know. Code string `json:"code"` // DocUrl Where the code is explained. @@ -7728,6 +7800,9 @@ type OfferedTool struct { // Description What the model is told the tool does. Description string `json:"description"` + // Loading How a tool is offered to the model. full sends its whole description and input schema from the start. summary sends the first line of its description and its arguments without their descriptions, and the first call returns the full description and schema instead of running it: one more model turn for each tool a conversation uses, for less context on every turn. deferred offers nothing until the model needs it: an agent with a deferred tool is offered search_tool, which finds deferred tools by what they do, and load_tool, which returns their full descriptions and offers them from the model's next step on. + Loading *ToolLoading `json:"loading,omitempty"` + // Name How the model asks for it. Name string `json:"name"` @@ -7958,7 +8033,7 @@ type Policy struct { // Example: {"application":"support"} Tags *map[string]string `json:"tags,omitempty"` - // ToolOutputRetention How long a session keeps what its tools were given and gave back on the turns it records: a tool's arguments, its result, a search's query and the wording of a tool's error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed, so a session's items read in the same order and tool usage still counts the call. Left out, or empty, it is kept as long as the session. The shortest of the organization's, the app's and the agent config's tools.output_retention applies, so an agent can shorten it and never lengthen it. It applies to tool calls recorded after it is set. Deleting a session deletes all of it at once. + // ToolOutputRetention How long a session keeps what its tools were given and gave back on the turns it records: a tool's arguments, its result, a search's query and the wording of a tool's error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed, so a session's items read in the same order and tool usage still counts the call. Left out, or empty, it is kept as long as the session. The shortest of the organization's, the app's and the output_retention of the agent config's tool group or connector binding the tool is in applies, so an agent can shorten it and never lengthen it. It applies to tool calls recorded after it is set. Deleting a session deletes all of it at once. // // Example: 30d ToolOutputRetention *string `json:"tool_output_retention,omitempty"` @@ -10241,9 +10316,9 @@ type SyncAgentRequest struct { Subagent *string `json:"subagent,omitempty"` Tags *map[string]string `json:"tags,omitempty"` - // Tools How an agent is offered its tools, and how long what they were given and gave back is kept. - Tools *AgentTools `json:"tools,omitempty"` - Tts *string `json:"tts,omitempty"` + // Tools agent.yaml's tools: the functions an SDK provides that the agent may offer, in named groups, as AgentConfigRequest's tools names them, with the same rules. Sent, they replace the stored groups whole, an empty list removing them. Left out, the stored ones stay. + Tools *[]ToolGroup `json:"tools,omitempty"` + Tts *string `json:"tts,omitempty"` // UiInstructions What teaches the agent's model to answer with UI, which the SDK compiles from agent.yaml's ui key. Kept apart from instructions, so a session opened with ui false is not told them. Left out, the agent is taught none. UiInstructions *string `json:"ui_instructions,omitempty"` @@ -10476,6 +10551,62 @@ type ToolApprovalCommand struct { // ToolApprovalCommandType defines model for ToolApprovalCommand.Type. type ToolApprovalCommandType string +// ToolCallFrameType tool_call.started is the call's first frame, tool_call.progress one while it runs or waits, and tool_call.finished its last. +type ToolCallFrameType string + +// ToolCallState One change to a tool call's state. Every watcher is sent tool_call.started once, tool_call.progress for each change while it runs or waits, and tool_call.finished once, carrying the call's whole state each time, so a client keeps the last one it got per tool_call_id. +type ToolCallState struct { + // ApprovedAt When a person allowed the call. + ApprovedAt *time.Time `json:"approved_at,omitempty"` + + // DurationMs From started_at to this change; on tool_call.finished, how long the call took. + DurationMs int64 `json:"duration_ms"` + + // Error Why a failed call failed, or what a declined one shows, such as "Not allowed in time". + Error *string `json:"error,omitempty"` + + // Progress The last thing said about the call while it runs, such as "Searching for flights": a tool_update, the connector binding's pre_speech, or the question it waits on. + Progress *string `json:"progress,omitempty"` + + // RejectedAt When a person declined the call, or its wait for them passed. + RejectedAt *time.Time `json:"rejected_at,omitempty"` + StartedAt time.Time `json:"started_at"` + + // Status in_progress while it runs, waiting while it is held for a person's approval, and success, failed or cancelled once it finished. A call declined, or whose approval nobody gave in time, is cancelled with rejected_at. + Status ToolCallStatus `json:"status"` + Tool string `json:"tool"` + + // ToolCallId The call, as tool_call, tool_started and tool_ran name it. + ToolCallId string `json:"tool_call_id"` + TurnId string `json:"turn_id"` + + // Type tool_call.started is the call's first frame, tool_call.progress one while it runs or waits, and tool_call.finished its last. + Type ToolCallFrameType `json:"type"` +} + +// ToolCallStatus in_progress while it runs, waiting while it is held for a person's approval, and success, failed or cancelled once it finished. A call declined, or whose approval nobody gave in time, is cancelled with rejected_at. +type ToolCallStatus string + +// ToolGroup A named set of functions an SDK provides that the agent may offer, how they are loaded and how long what they were given and gave back is kept. A function an SDK registers, declares on a session or hosts in a worker is offered only when a group names it. +type ToolGroup struct { + // Loading How a tool is offered to the model. full sends its whole description and input schema from the start. summary sends the first line of its description and its arguments without their descriptions, and the first call returns the full description and schema instead of running it: one more model turn for each tool a conversation uses, for less context on every turn. deferred offers nothing until the model needs it: an agent with a deferred tool is offered search_tool, which finds deferred tools by what they do, and load_tool, which returns their full descriptions and offers them from the model's next step on. + Loading *ToolLoading `json:"loading,omitempty"` + + // Name What the group is called, unique within the config. + Name string `json:"name"` + + // OutputRetention How long the agent's sessions keep what the group's tools were given and gave back on the turns they record: a tool's arguments, its result and the wording of its error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed. Left out or empty, it is kept as long as the session. The app's and the organization's policies (tool_output_retention) can shorten it and never lengthen it. It applies to tool calls recorded after it is set. + // + // Example: 30d + OutputRetention *string `json:"output_retention,omitempty"` + + // Tools The functions' names, as the SDK registers them. A name sits in one group at most. + Tools []string `json:"tools"` +} + +// ToolLoading How a tool is offered to the model. full sends its whole description and input schema from the start. summary sends the first line of its description and its arguments without their descriptions, and the first call returns the full description and schema instead of running it: one more model turn for each tool a conversation uses, for less context on every turn. deferred offers nothing until the model needs it: an agent with a deferred tool is offered search_tool, which finds deferred tools by what they do, and load_tool, which returns their full descriptions and offers them from the model's next step on. +type ToolLoading string + // ToolResultCommand defines model for ToolResultCommand. type ToolResultCommand struct { Error *string `json:"error,omitempty"` @@ -10496,12 +10627,30 @@ type ToolStats struct { // AvgTokens What a call put in front of the model, its arguments and its result, estimated at four characters a token. AvgTokens float64 `json:"avg_tokens"` - Calls int64 `json:"calls"` + + // AvgTurnsBeforeUse Turns a session took before its first call, on average: 0 is a call on the first turn. Null on the row over every tool. + AvgTurnsBeforeUse *float64 `json:"avg_turns_before_use"` + Calls int64 `json:"calls"` + + // DefinitionTokens What the tool's whole definition, its name, description and schema, costs every turn that carries it, estimated at four characters a token. Null when no call recorded it. + DefinitionTokens *int64 `json:"definition_tokens"` // Failed Calls that ended in an error. - Failed int64 `json:"failed"` - P99Ms *float64 `json:"p99_ms"` - Tool string `json:"tool"` + Failed int64 `json:"failed"` + + // Loading How the latest call had the tool offered: full, summary or deferred. Empty when no call recorded it, as calls made before it was recorded and built-in tools. + Loading string `json:"loading"` + P99Ms *float64 `json:"p99_ms"` + + // SessionShare Sessions that called it, out of the window's sessions, from 0 to 1. Null when no session took a turn. + SessionShare *float64 `json:"session_share"` + + // Sessions Sessions that called it at least once. + Sessions int64 `json:"sessions"` + + // SummaryTokens What its definition costs offered as a summary. Null unless its loading is summary. + SummaryTokens *int64 `json:"summary_tokens"` + Tool string `json:"tool"` } // TranscriptEntity Something the recording named, for the providers that pick them out. @@ -11878,6 +12027,9 @@ type GetAgentStatsParams struct { // To End of the window, exclusive. To time.Time `form:"to" json:"to"` + + // Agent The id of an agent config: only its sessions are counted. Omitted is every agent. + Agent *string `form:"agent,omitempty" json:"agent,omitempty"` } // GetSpendParams defines parameters for GetSpend. @@ -16116,7 +16268,8 @@ type ClientInterface interface { // GetAgentStats How the calling customer's agents ran, across all of them // - // The most agents that were running at once, and which tools they called: how often, how much each call put in front of the model and how long it took. + // The most agents that were running at once, and which tools they called: how often, in how many sessions and after how many turns, how much each call put in front of the model, what the tool's definition costs every turn that carries it and how long a call took. + // With agent, the sessions, turns and tools are that agent config's alone; the peak is still every agent's. // One figure for the whole window rather than a row per bucket, because neither a peak nor a percentile can be summed out of smaller ones. // // Corresponds with GET /v1/stats/agents (the `GetAgentStats` operationId). @@ -22822,7 +22975,8 @@ func (c *Client) GetActivity(ctx context.Context, params *GetActivityParams, req // GetAgentStats How the calling customer's agents ran, across all of them // -// The most agents that were running at once, and which tools they called: how often, how much each call put in front of the model and how long it took. +// The most agents that were running at once, and which tools they called: how often, in how many sessions and after how many turns, how much each call put in front of the model, what the tool's definition costs every turn that carries it and how long a call took. +// With agent, the sessions, turns and tools are that agent config's alone; the peak is still every agent's. // One figure for the whole window rather than a row per bucket, because neither a peak nor a percentile can be summed out of smaller ones. // // Corresponds with GET /v1/stats/agents (the `GetAgentStats` operationId). @@ -34850,6 +35004,18 @@ func NewGetAgentStatsRequest(server string, params *GetAgentStatsParams) (*http. } } + if params.Agent != nil { + + if queryFrag, err := runtime.StyleParamWithOptions("form", false, "agent", *params.Agent, runtime.StyleParamOptions{ParamLocation: runtime.ParamLocationQuery, Type: "string", Format: ""}); err != nil { + return nil, err + } else { + for _, qp := range strings.Split(queryFrag, "&") { + rawQueryFragments = append(rawQueryFragments, qp) + } + } + + } + if encoded := queryValues.Encode(); encoded != "" { rawQueryFragments = append(rawQueryFragments, encoded) } @@ -39430,7 +39596,8 @@ type ClientWithResponsesInterface interface { // GetAgentStatsWithResponse How the calling customer's agents ran, across all of them // - // The most agents that were running at once, and which tools they called: how often, how much each call put in front of the model and how long it took. + // The most agents that were running at once, and which tools they called: how often, in how many sessions and after how many turns, how much each call put in front of the model, what the tool's definition costs every turn that carries it and how long a call took. + // With agent, the sessions, turns and tools are that agent config's alone; the peak is still every agent's. // One figure for the whole window rather than a row per bucket, because neither a peak nor a percentile can be summed out of smaller ones. // // Returns a wrapper object for the known response body format(s). @@ -65354,6 +65521,8 @@ type GetAgentStatsResponse struct { JSON402 *PaymentRequired // JSON403 the response for an HTTP 403 `application/json` response JSON403 *Forbidden + // JSON404 the response for an HTTP 404 `application/json` response + JSON404 *NotFound // JSON500 the response for an HTTP 500 `application/json` response JSON500 *InternalError // JSON503 the response for an HTTP 503 `application/json` response @@ -65389,6 +65558,11 @@ func (r GetAgentStatsResponse) GetJSON403() *Forbidden { return r.JSON403 } +// GetJSON404 returns the response for an HTTP 404 `application/json` response +func (r GetAgentStatsResponse) GetJSON404() *NotFound { + return r.JSON404 +} + // GetJSON500 returns the response for an HTTP 500 `application/json` response func (r GetAgentStatsResponse) GetJSON500() *InternalError { return r.JSON500 @@ -72465,7 +72639,8 @@ func (c *ClientWithResponses) GetActivityWithResponse(ctx context.Context, param // GetAgentStatsWithResponse How the calling customer's agents ran, across all of them // -// The most agents that were running at once, and which tools they called: how often, how much each call put in front of the model and how long it took. +// The most agents that were running at once, and which tools they called: how often, in how many sessions and after how many turns, how much each call put in front of the model, what the tool's definition costs every turn that carries it and how long a call took. +// With agent, the sessions, turns and tools are that agent config's alone; the peak is still every agent's. // One figure for the whole window rather than a row per bucket, because neither a peak nor a percentile can be summed out of smaller ones. // // Returns a wrapper object for the known response body format(s). @@ -102808,6 +102983,13 @@ func ParseGetAgentStatsResponse(rsp *http.Response) (*GetAgentStatsResponse, err } response.JSON403 = &dest + case strings.Contains(rsp.Header.Get("Content-Type"), "json") && rsp.StatusCode == 404: + var dest NotFound + if err := json.Unmarshal(bodyBytes, &dest); err != nil { + return nil, err + } + response.JSON404 = &dest + case strings.Contains(rsp.Header.Get("Content-Type"), "json") && rsp.StatusCode == 500: var dest InternalError if err := json.Unmarshal(bodyBytes, &dest); err != nil { diff --git a/sdks/go/agents/agent_test.go b/sdks/go/agents/agent_test.go index 8aec06efda..26af722e5e 100644 --- a/sdks/go/agents/agent_test.go +++ b/sdks/go/agents/agent_test.go @@ -1219,9 +1219,11 @@ func TestSyncSendsNoChannelsWhenTheFileNamesNone(t *testing.T) { } } -func TestSyncSendsWhetherToolsAreOfferedProgressively(t *testing.T) { +func TestSyncSendsTheToolGroups(t *testing.T) { root := filepath.Join(t.TempDir(), "concierge") - write(t, root, "agent.yaml", "tools:\n progressive: true\n") + write(t, root, "agent.yaml", "tools:\n"+ + " - name: booking\n tools: [lookup_booking]\n"+ + " - name: refunds\n loading: deferred\n output_retention: 30d\n tools: [refund_booking, void_ticket]\n") router := newBackend(t) agent := agentOn(t, router, Options{Dir: root}) @@ -1231,14 +1233,24 @@ func TestSyncSendsWhetherToolsAreOfferedProgressively(t *testing.T) { router.mu.Lock() defer router.mu.Unlock() - if sent := router.syncs[0].Tools; sent == nil || sent.Progressive == nil || !*sent.Progressive { - t.Errorf("tools went as %+v", sent) + sent := router.syncs[0].Tools + if sent == nil || len(*sent) != 2 { + t.Fatalf("tools went as %+v", sent) + } + booking, refunds := (*sent)[0], (*sent)[1] + if booking.Name != "booking" || booking.Loading != nil || booking.OutputRetention != nil { + t.Errorf("a group that says nothing more went as %+v", booking) + } + if refunds.Loading == nil || *refunds.Loading != acceleration.ToolLoadingDeferred || + refunds.OutputRetention == nil || *refunds.OutputRetention != "30d" || + !slices.Equal(refunds.Tools, []string{"refund_booking", "void_ticket"}) { + t.Errorf("refunds went as %+v", refunds) } } -func TestSyncSendsHowLongToolOutputIsKept(t *testing.T) { +func TestSyncSendsHowAConnectorsToolsAreOffered(t *testing.T) { root := filepath.Join(t.TempDir(), "concierge") - write(t, root, "agent.yaml", "tools:\n output_retention: 30d\n") + write(t, root, "agent.yaml", "connectors:\n - name: linear\n loading: summary\n output_retention: none\n") router := newBackend(t) agent := agentOn(t, router, Options{Dir: root}) @@ -1248,8 +1260,14 @@ func TestSyncSendsHowLongToolOutputIsKept(t *testing.T) { router.mu.Lock() defer router.mu.Unlock() - if sent := router.syncs[0].Tools; sent == nil || sent.OutputRetention == nil || *sent.OutputRetention != "30d" { - t.Errorf("tools went as %+v", sent) + bindings := router.syncs[0].Connectors + if bindings == nil || len(*bindings) != 1 { + t.Fatalf("connectors went as %+v", bindings) + } + binding := (*bindings)[0] + if binding.Loading == nil || *binding.Loading != acceleration.ToolLoadingSummary || + binding.OutputRetention == nil || *binding.OutputRetention != "none" { + t.Errorf("linear went as %+v", binding) } } diff --git a/sdks/go/agents/config.go b/sdks/go/agents/config.go index 22a3af21d0..b4a276e41b 100644 --- a/sdks/go/agents/config.go +++ b/sdks/go/agents/config.go @@ -275,9 +275,19 @@ func declareSettings(body *acceleration.SyncAgentRequest, settings Settings) { } } if settings.Tools != nil { - body.Tools = &acceleration.AgentTools{ - Progressive: settings.Tools.Progressive, OutputRetention: settings.Tools.OutputRetention, + groups := make([]acceleration.ToolGroup, 0, len(*settings.Tools)) + for _, group := range *settings.Tools { + sent := acceleration.ToolGroup{Name: group.Name, Tools: group.Tools} + if group.Loading != "" { + loading := acceleration.ToolLoading(group.Loading) + sent.Loading = &loading + } + if group.OutputRetention != "" { + sent.OutputRetention = &group.OutputRetention + } + groups = append(groups, sent) } + body.Tools = &groups } if settings.Channels != nil { declared := acceleration.AgentChannels{ @@ -479,6 +489,13 @@ func connectorBinding(ctx context.Context, client *acceleration.ClientWithRespon Connection: acceleration.AgentConnectorSelection{Type: acceleration.AgentConnectorSelectionTypeSession}, } binding.RequireApproval = entry.RequireApproval + if entry.Loading != "" { + loading := acceleration.ToolLoading(entry.Loading) + binding.Loading = &loading + } + if entry.OutputRetention != "" { + binding.OutputRetention = &entry.OutputRetention + } if entry.Sandbox { binding.Sandbox = &entry.Sandbox } diff --git a/sdks/go/agents/dispatch.go b/sdks/go/agents/dispatch.go index 780596965e..13ac801a44 100644 --- a/sdks/go/agents/dispatch.go +++ b/sdks/go/agents/dispatch.go @@ -155,14 +155,15 @@ func (d *Dispatch) OnCall(handler func(context.Context, InboundCall) error) { d.dispatch.OnCall(handler) } -// Host runs an agent's tools for every session opened under it, whoever opened it, with -// toolTimeout as how long the router waits for each tool call. See stream.Dispatch.Host. +// Host runs the app's tools for every agent whose tool groups name them, whoever opened the +// session, with toolTimeout as how long the router waits for each tool call. See +// stream.Dispatch.Host. client.RunDispatch does this alone, for a worker with no calls or +// messages to answer. // -// agent := api.Agent("my-agent") -// agent.Tools().Add(LookupOrder{}) -// dispatch.Host(agent, time.Minute) -func (d *Dispatch) Host(agent stream.HostedAgent, toolTimeout time.Duration) { - d.dispatch.Host(agent, toolTimeout) +// api.Tools().Add(LookupOrder{}) +// dispatch.Host(api, time.Minute) +func (d *Dispatch) Host(host stream.ToolHost, toolTimeout time.Duration) { + d.dispatch.Host(host, toolTimeout) } // WorkerID is what the router calls this connection. diff --git a/sdks/go/agents/dispatch_test.go b/sdks/go/agents/dispatch_test.go index 3107a7594b..a1bb080507 100644 --- a/sdks/go/agents/dispatch_test.go +++ b/sdks/go/agents/dispatch_test.go @@ -691,9 +691,9 @@ func TestAMessageARunningSessionHoldsIsNotGivenASecondAgent(t *testing.T) { } } -func TestAClientAgentsToolsAreHostedUnderItsName(t *testing.T) { - // The router matches a hosted tool on a session's agent id or its agent's name, and the - // name is what a caller holding the handle knows the agent as. +func TestTheClientsToolsAreHostedForEveryAgent(t *testing.T) { + // Registered once for the app, the router offers each to the agents whose tool groups + // name it, so the worker names no agent. told := make(chan stream.Frame, 2) router := newWorked(t, func(connection *websocket.Conn) { for { @@ -716,21 +716,20 @@ func TestAClientAgentsToolsAreHostedUnderItsName(t *testing.T) { if err != nil { t.Fatal(err) } - agent := api.Agent("stream-support") var ran atomic.Bool - if err := agent.Tools().Add(count{ran: &ran}); err != nil { + if err := api.Tools().Add(count{ran: &ran}); err != nil { t.Fatal(err) } dispatch, err := NewDispatch(DispatchOptions{Backend: backend, Logger: slog.New(slog.DiscardHandler)}) if err != nil { t.Fatal(err) } - dispatch.Host(agent, time.Minute) + dispatch.Host(api, time.Minute) stop := waited(t, dispatch) defer stop() declared, answered := <-told, <-told - if declared.String("agent_id") != "stream-support" || declared.Int("timeout_ms") != 60000 { + if declared["every_agent"] != true || declared.Int("timeout_ms") != 60000 { t.Errorf("the router was told %v", declared) } if answered.String("output") != "42" || !ran.Load() { @@ -738,6 +737,50 @@ func TestAClientAgentsToolsAreHostedUnderItsName(t *testing.T) { } } +func TestRunDispatchRunsTheClientsToolsUntilItsContextEnds(t *testing.T) { + told := make(chan stream.Frame, 2) + router := newWorked(t, func(connection *websocket.Conn) { + for { + var frame stream.Frame + if err := connection.ReadJSON(&frame); err != nil { + return + } + switch frame.Type() { + case "host_tools": + told <- frame + _ = connection.WriteJSON(stream.Frame{"type": "tool_call", "id": "call-1", "name": "count", "arguments": `{}`}) + case "tool_result": + told <- frame + } + } + }) + api, err := client.New(stream.Backend{URL: router.URL, CustomerID: "acme"}) + if err != nil { + t.Fatal(err) + } + var ran atomic.Bool + if err := api.Tools().Add(count{ran: &ran}); err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithCancel(t.Context()) + stopped := make(chan error, 1) + go func() { stopped <- api.RunDispatch(ctx) }() + + declared, answered := <-told, <-told + if declared["every_agent"] != true { + t.Errorf("the router was told %v", declared) + } + if answered.String("output") != "42" || !ran.Load() { + t.Errorf("the hosted tool was answered %v", answered) + } + cancel() + select { + case <-stopped: + case <-time.After(5 * time.Second): + t.Fatal("RunDispatch kept running after its context ended") + } +} + func isClosed(done chan struct{}) bool { select { case <-done: diff --git a/sdks/go/agents/folder.go b/sdks/go/agents/folder.go index ba0d9bab90..d3d02f8aec 100644 --- a/sdks/go/agents/folder.go +++ b/sdks/go/agents/folder.go @@ -197,9 +197,10 @@ type Settings struct { // file lists none, which removes them. A null entry is refused: dropping it // would sync an empty list and wipe the stored bindings. Connectors connectorList `yaml:"connectors"` - // Tools is how MCP server and connector tools are offered. Nil when the - // declaration says nothing about it. - Tools *ToolSettings `yaml:"tools"` + // Tools are the groups naming the functions this process provides that the agent may + // offer: a function no group names is never offered. Nil when the declaration says + // nothing, which leaves the groups stored. + Tools *toolGroupList `yaml:"tools"` // Channels are the lines the agent answers on besides its Stream Chat channel: a // WhatsApp number, a number to text, an iMessage line. Each names a number the app // connected on the router, which is where the provider's credentials live. @@ -234,15 +235,66 @@ type Settings struct { App map[string]any `yaml:"app"` } -// ToolSettings is how MCP server and connector tools are offered, and how long what any tool -// was given and gave back is kept. -type ToolSettings struct { - // Progressive offers each tool by a summary, the first call to it returning its full - // description instead of running it. Nil when the declaration says nothing about it. - Progressive *bool `yaml:"progressive"` - // OutputRetention is how long the agent's sessions keep a tool's arguments and output: - // none, or a duration such as 12h or 30d. Nil when the declaration says nothing about it. - OutputRetention *string `yaml:"output_retention"` +// ToolGroupSettings is one group of the agent's functions, and how they are offered. +type ToolGroupSettings struct { + Name string `yaml:"name"` + // Loading is full, the whole description and schema; summary, the first line, with the + // first call returning the rest; or deferred, offered only once the model finds and loads + // it. Empty is full. + Loading string `yaml:"loading"` + // Tools are the names of the functions in the group. + Tools []string `yaml:"tools"` + // OutputRetention is how long the agent's sessions keep what the group's tools were + // given and gave back: none, or a duration such as 12h or 30d. Empty is as long as the + // session. + OutputRetention string `yaml:"output_retention"` +} + +// toolLoadings are what a group's or a connector's loading may say. +var toolLoadings = []string{"full", "summary", "deferred"} + +// toolGroupList is the tools list. The mapping tools was before groups is refused with where +// each of its settings went. +type toolGroupList []ToolGroupSettings + +func (t *toolGroupList) UnmarshalYAML(node *yaml.Node) error { + if node.Kind == yaml.MappingNode { + return fmt.Errorf("line %d: tools is a list of groups, each with a name and its tools, such as "+ + "- {name: booking, loading: full, tools: [lookup_booking]}; progressive: true is now "+ + "loading: summary, and output_retention moves onto each group and connector", node.Line) + } + if node.Kind != yaml.SequenceNode { + return fmt.Errorf("line %d: tools must be a list of groups", node.Line) + } + keys := []string{"name", "loading", "tools", "output_retention"} + groups := make(toolGroupList, 0, len(node.Content)) + for _, item := range node.Content { + if item.Kind != yaml.MappingNode { + return fmt.Errorf("line %d: a tool group is a mapping with a name and its tools", item.Line) + } + for index := 0; index < len(item.Content); index += 2 { + if key := item.Content[index]; !slices.Contains(keys, key.Value) { + return fmt.Errorf("line %d: %q is not something a tool group says; %s are", key.Line, key.Value, + strings.Join(keys, ", ")) + } + } + var group ToolGroupSettings + if err := item.Decode(&group); err != nil { + return err + } + switch { + case group.Name == "": + return fmt.Errorf("line %d: a tool group needs a name", item.Line) + case len(group.Tools) == 0: + return fmt.Errorf("line %d: tool group %q names no tools", item.Line, group.Name) + case group.Loading != "" && !slices.Contains(toolLoadings, group.Loading): + return fmt.Errorf("line %d: tool group %q: loading is %s, not %q", keyLine(item, "loading"), group.Name, + strings.Join(toolLoadings, ", "), group.Loading) + } + groups = append(groups, group) + } + *t = groups + return nil } // ConnectorSettings is one connector the agent binds. agent.yaml gives it as the alias @@ -267,6 +319,13 @@ type ConnectorSettings struct { // RequireApproval are names or path.Match patterns of the tools whose every call waits for // the session's user to allow it. Nil sends no require_approval key. RequireApproval *[]string `yaml:"require_approval"` + // Loading is how the connector's tools are offered, as a tool group's: full, summary or + // deferred. Empty is full. + Loading string `yaml:"loading"` + // OutputRetention is how long the agent's sessions keep what the connector's tools were + // given and gave back: none, or a duration such as 12h or 30d. Empty is as long as the + // session. + OutputRetention string `yaml:"output_retention"` } // connectorKeys are the keys a connector entry may hold: the yaml tags of ConnectorSettings. @@ -308,6 +367,10 @@ func (c *ConnectorSettings) UnmarshalYAML(node *yaml.Node) error { if c.Connection != nil && *c.Connection == "" { return fmt.Errorf("line %d: connector %q: connection must be a connection id or label", keyLine(node, "connection"), c.Name) } + if c.Loading != "" && !slices.Contains(toolLoadings, c.Loading) { + return fmt.Errorf("line %d: connector %q: loading is %s, not %q", keyLine(node, "loading"), c.Name, + strings.Join(toolLoadings, ", "), c.Loading) + } return nil } diff --git a/sdks/go/agents/folder_test.go b/sdks/go/agents/folder_test.go index c911174c30..ded5276161 100644 --- a/sdks/go/agents/folder_test.go +++ b/sdks/go/agents/folder_test.go @@ -401,6 +401,39 @@ func TestTheOldPluginKeysAreRefused(t *testing.T) { } } +func TestTheToolSettingsOfBeforeGroupsAreRefusedSayingWhereTheyWent(t *testing.T) { + for _, declared := range []string{"tools:\n progressive: true\n", "tools:\n output_retention: 30d\n"} { + root := filepath.Join(t.TempDir(), "concierge") + write(t, root, "agent.yaml", declared) + + _, err := Load(root) + + if err == nil || !strings.Contains(err.Error(), "loading: summary") || !strings.Contains(err.Error(), "output_retention moves") { + t.Errorf("%q: %v", declared, err) + } + } +} + +func TestAToolGroupThatSaysTooLittleOrTooMuchIsRefused(t *testing.T) { + for declared, says := range map[string]string{ + "tools:\n - tools: [lookup_booking]\n": "needs a name", + "tools:\n - name: booking\n": "names no tools", + "tools:\n - name: booking\n loading: lazy\n tools: [a]\n": `not "lazy"`, + "tools:\n - name: booking\n progressive: true\n tools: [a]\n": `"progressive" is not something a tool group says`, + "tools:\n - lookup_booking\n": "a tool group is a mapping", + "connectors:\n - name: linear\n loading: eager\n": `connector "linear": loading`, + } { + root := filepath.Join(t.TempDir(), "concierge") + write(t, root, "agent.yaml", declared) + + _, err := Load(root) + + if err == nil || !strings.Contains(err.Error(), says) { + t.Errorf("%q: %v", declared, err) + } + } +} + func TestADeclarationKeyNobodyKnowsIsRefused(t *testing.T) { for _, declaration := range []string{ "name: jean\nlmm: openai/gpt-5.6\n", diff --git a/sdks/go/client/client.go b/sdks/go/client/client.go index 93728a65c4..6b38ff60a0 100644 --- a/sdks/go/client/client.go +++ b/sdks/go/client/client.go @@ -26,6 +26,7 @@ import ( "net/http" "os" "sync" + "time" getstream "github.com/GetStream/getstream-go/v5" @@ -39,6 +40,8 @@ type Client struct { backend stream.Backend // watcher reads what every answer says about money (WithUsage, WithBalanceWarning). watcher *watcher + // tools are the app's, registered once for every agent and run by RunDispatch. + tools *tools.Registry // mu guards the Stream client, which is built on the first conversation that asks for // chat or video and shared by every one after it. @@ -70,10 +73,10 @@ func New(backend stream.Backend, options ...Option) (*Client, error) { // Only the transport the first client set is recognised: one wrapped again around it, // for tracing say, is read twice. if heard := heardBy(resolved.HTTPClient); heard != nil && watching.usage == nil && watching.warned == nil { - return &Client{backend: resolved, watcher: heard}, nil + return &Client{backend: resolved, watcher: heard, tools: tools.NewRegistry()}, nil } resolved.HTTPClient = watching.transport(resolved.HTTPClient) - return &Client{backend: resolved, watcher: watching}, nil + return &Client{backend: resolved, watcher: watching, tools: tools.NewRegistry()}, nil } // SetUser says who this client is acting for, and hands over the token that proves it. @@ -99,7 +102,7 @@ func (c *Client) SetUser(userID, token string) (*Client, error) { if err != nil { return nil, err } - return &Client{backend: resolved, watcher: c.watcher}, nil + return &Client{backend: resolved, watcher: c.watcher, tools: c.tools}, nil } // ServerSide says whether this client speaks for the app itself rather than for one user. @@ -114,7 +117,7 @@ func (c *Client) ServerSide() bool { return c.backend.UserID == "" } // No request is made: this is the name in a wrapper, and a name that matches nothing // configured is refused when a conversation is opened rather than here. func (c *Client) Agent(name string) *Agent { - agent := &Agent{client: c, name: name, tools: tools.NewRegistry()} + agent := &Agent{client: c, name: name} agent.Sessions = &Sessions{client: c, agent: agent} return agent } @@ -122,6 +125,26 @@ func (c *Client) Agent(name string) *Agent { // Backend is where this client points, for the parts of the SDK that take one directly. func (c *Client) Backend() stream.Backend { return c.backend } +// Tools are the app's tools, registered once for every agent. An agent is offered one only +// when a tool group of its agent.yaml names it, and its calls run wherever RunDispatch is +// running: +// +// api.Tools().Add(GetWeather{}) +// err := api.RunDispatch(ctx) +func (c *Client) Tools() *tools.Registry { return c.tools } + +// RunDispatch runs the app's tools for every session they are offered to, whoever opened it, +// until ctx is done. A tool's own timeout (tools.Timeout) is how long the router waits for +// one of its calls; without one it waits 30 seconds. +func (c *Client) RunDispatch(ctx context.Context) error { + dispatch, err := stream.NewDispatch(stream.DispatchOptions{Backend: c.backend}) + if err != nil { + return err + } + dispatch.Host(c, 0) + return dispatch.Run(ctx) +} + // api is the generated client, carrying this client's credentials. func (c *Client) api() (*acceleration.ClientWithResponses, error) { return c.backend.Client() @@ -167,22 +190,11 @@ type Agent struct { client *Client name string - // tools are the caller's own, offered by every conversation this agent opens. Held on - // the agent rather than per session because a tool added once should not have to be - // added again for the next conversation. - tools *tools.Registry } // Name is what the agent is called, which is what a caller knows it as. func (a *Agent) Name() string { return a.name } -// Tools are the ones this agent's conversations offer the model, to add to, and what a -// dispatch worker runs for every session of this agent once it hosts them with -// dispatch.Host(agent, timeout): -// -// agent.Tools().Add(GetWeather{}) -func (a *Agent) Tools() *tools.Registry { return a.tools } - // Config is how the agent is configured, as the backend has it, or nil for a name nothing is // stored under. // @@ -237,6 +249,35 @@ func (a *Agent) UpdateConfig(ctx context.Context, patch acceleration.AgentConfig return patched.JSON200, nil } +// Stats is how the agent's sessions used their tools from from up to to: how many sessions +// took a turn and how many turns, and per tool its calls, failures, latency, share of +// sessions, turns before its first use and what its definition costs every turn. +// +// Server side only, like the config it reads the agent's id from. +func (a *Agent) Stats(ctx context.Context, from, to time.Time) (*acceleration.AgentStats, error) { + config, err := a.Config(ctx) + if err != nil { + return nil, err + } + if config == nil { + return nil, fmt.Errorf("client: there is no agent called %s to read the stats of", a.name) + } + + api, err := a.client.api() + if err != nil { + return nil, err + } + read, err := api.GetAgentStatsWithResponse(ctx, + &acceleration.GetAgentStatsParams{From: from, To: to, Agent: &config.Id}) + if err != nil { + return nil, fmt.Errorf("client: reading the stats of the agent %s: %w", a.name, err) + } + if read.JSON200 == nil { + return nil, failure("reading the stats of the agent "+a.name, read.HTTPResponse, read.Body) + } + return read.JSON200, nil +} + // failure is what the router said went wrong with what was being done, read off the answer // it gave instead, or the status when it said nothing. func failure(what string, response *http.Response, body []byte) error { diff --git a/sdks/go/client/client_test.go b/sdks/go/client/client_test.go index 2a49b5ba10..4c320282cc 100644 --- a/sdks/go/client/client_test.go +++ b/sdks/go/client/client_test.go @@ -89,6 +89,18 @@ func newRouter(t *testing.T) *router { answer(w, http.StatusNotFound, acceleration.ErrorResponse{Error: acceleration.ErrorDetail{Message: "no such agent config"}}) }) + mux.HandleFunc("GET /v1/stats/agents", func(w http.ResponseWriter, r *http.Request) { + backend.record(r) + if r.URL.Query().Get("agent") != "config-1" { + answer(w, http.StatusNotFound, acceleration.ErrorResponse{Error: acceleration.ErrorDetail{Message: "no such agent config"}}) + return + } + answer(w, http.StatusOK, acceleration.AgentStats{ + Sessions: 40, AvgTurns: ptr(6.5), + Tools: []acceleration.ToolStats{{Tool: "weather_lookup", Calls: 12, Sessions: 10, SessionShare: ptr(0.25)}}, + }) + }) + mux.HandleFunc("GET /v1/settings/app", func(w http.ResponseWriter, r *http.Request) { backend.record(r) answer(w, http.StatusOK, acceleration.AppSettings{Stream: acceleration.StreamSettings{ @@ -1943,29 +1955,6 @@ func TestAResumesToolTimeoutGoesOverInMilliseconds(t *testing.T) { } } -func TestAResumeDeclaresTheAgentsFunctions(t *testing.T) { - backend := newRouter(t) - agent := backend.client(t).Agent("docs") - if err := register(agent.Tools(), "get_weather"); err != nil { - t.Fatal(err) - } - - resumed, err := agent.Sessions.Resume(t.Context(), "session-7") - if err != nil { - t.Fatal(err) - } - defer resumed.Close(t.Context()) - - body := backend.body(t, "POST", "/v1/agents/sessions/session-7/resume") - declared, _ := body["tools"].([]any) - if len(declared) != 1 || declared[0].(map[string]any)["name"] != "get_weather" { - t.Errorf("tools went over as %v", body["tools"]) - } - if _, sent := body["tool_timeout_ms"]; sent { - t.Errorf("tool_timeout_ms went over as %v though none was given", body["tool_timeout_ms"]) - } -} - func TestAResumeWithNothingToSayReadsTheConversationAsItIs(t *testing.T) { // A timeout under a millisecond would go over as zero, 30 seconds, not what it ran with. backend := newRouter(t) @@ -2003,11 +1992,12 @@ func TestForkingWithoutMessagesSaysSo(t *testing.T) { func TestAForkInheritsTheParentsFunctions(t *testing.T) { backend := newRouter(t) agent := backend.client(t).Agent("docs") - if err := register(agent.Tools(), "get_weather"); err != nil { + own := tools.NewRegistry() + if err := register(own, "get_weather"); err != nil { t.Fatal(err) } - session, err := agent.Sessions.Create(t.Context(), SessionOptions{}) + session, err := agent.Sessions.Create(t.Context(), SessionOptions{Tools: own}) if err != nil { t.Fatal(err) } @@ -2048,12 +2038,13 @@ func (s *SessionToolsSuite) TestAForkRunsTheToolsTheSessionHasNow() { }} agent := s.backend.client(s.T()).Agent("docs") var withdrawnRan atomic.Bool - s.Require().NoError(tools.Register(agent.Tools(), "get_weather", "What the weather is", + own := tools.NewRegistry() + s.Require().NoError(tools.Register(own, "get_weather", "What the weather is", func(context.Context, struct{}) (any, error) { withdrawnRan.Store(true) return "raining", nil })) - session, err := agent.Sessions.Create(s.T().Context(), SessionOptions{}) + session, err := agent.Sessions.Create(s.T().Context(), SessionOptions{Tools: own}) s.Require().NoError(err) defer session.Close(s.T().Context()) added := tools.NewRegistry() @@ -2075,14 +2066,20 @@ func (s *SessionToolsSuite) TestAForkRunsTheToolsTheSessionHasNow() { s.False(withdrawnRan.Load(), "the fork ran a tool the session had withdrawn") } -func TestTheModelIsOfferedTheAgentsFunctions(t *testing.T) { +func TestAConversationDeclaresItsOwnToolsAndNotTheApps(t *testing.T) { + // The app's tools are for every agent: declared on a session, the ones its agent's + // tool groups do not name would have the router refuse it. backend := newRouter(t) - agent := backend.client(t).Agent("docs") - if err := register(agent.Tools(), "get_weather"); err != nil { + api := backend.client(t) + if err := register(api.Tools(), "get_weather"); err != nil { + t.Fatal(err) + } + own := tools.NewRegistry() + if err := register(own, "book_table"); err != nil { t.Fatal(err) } - session, err := agent.Sessions.Create(t.Context(), SessionOptions{}) + session, err := api.Agent("docs").Sessions.Create(t.Context(), SessionOptions{Tools: own}) if err != nil { t.Fatal(err) } @@ -2092,7 +2089,7 @@ func TestTheModelIsOfferedTheAgentsFunctions(t *testing.T) { if len(declared) != 1 { t.Fatalf("the session declared %v", declared) } - if tool, _ := declared[0].(map[string]any); tool["name"] != "get_weather" { + if tool, _ := declared[0].(map[string]any); tool["name"] != "book_table" { t.Errorf("the declared tool is %v", declared[0]) } } @@ -2147,6 +2144,37 @@ func TestUpdatingAnAgentNothingIsStoredUnderFails(t *testing.T) { } } +func TestAnAgentsStatsAreReadForItsConfigOverTheWindow(t *testing.T) { + backend := newRouter(t) + backend.configs = []acceleration.AgentConfig{{Id: "config-1", Name: "docs"}} + from := time.Date(2026, 11, 1, 0, 0, 0, 0, time.UTC) + to := from.AddDate(0, 1, 0) + + stats, err := backend.client(t).Agent("docs").Stats(t.Context(), from, to) + if err != nil { + t.Fatal(err) + } + if stats.Sessions != 40 || len(stats.Tools) != 1 || stats.Tools[0].Tool != "weather_lookup" { + t.Fatalf("the stats came back as %+v", stats) + } + asked := backend.query(t, "GET", "/v1/stats/agents") + if asked.Get("from") != "2026-11-01T00:00:00Z" || asked.Get("to") != "2026-12-01T00:00:00Z" { + t.Errorf("the window went over as %v", asked) + } +} + +func TestTheStatsOfAnAgentNothingIsStoredUnderFail(t *testing.T) { + backend := newRouter(t) + + _, err := backend.client(t).Agent("nowhere").Stats(t.Context(), time.Now().Add(-time.Hour), time.Now()) + if err == nil || !strings.Contains(err.Error(), "nowhere") { + t.Errorf("reading the stats of an agent nobody stored answered %v", err) + } + if backend.requests("GET", "/v1/stats/agents") != 0 { + t.Error("an agent with no config had every agent's stats read for it") + } +} + func TestANameNothingIsStoredUnderIsNoConfigRatherThanAFailure(t *testing.T) { backend := newRouter(t) diff --git a/sdks/go/client/sessions.go b/sdks/go/client/sessions.go index 2322235505..ba3140b06d 100644 --- a/sdks/go/client/sessions.go +++ b/sdks/go/client/sessions.go @@ -81,8 +81,8 @@ type SessionOptions struct { // leaves the user's stored zone, then the agent's default, to decide. Timezone string - // Tools are the caller's own, for this conversation only. Nil uses the agent's, which is - // the usual arrangement. + // Tools are the caller's own, for this conversation only, run in this process. The app's + // tools (Client.Tools) reach a conversation through RunDispatch instead. Tools *tools.Registry // Logger is where the session reports what it could not do. Nil uses the default. Logger *slog.Logger @@ -162,9 +162,6 @@ type Sessions struct { // conversation somebody comes back to usually is. func (s *Sessions) Create(ctx context.Context, options SessionOptions) (*Session, error) { functions := options.Tools - if functions == nil { - functions = s.agent.tools - } // The pipeline is what holds the socket and runs the caller's functions, so this builds // one rather than opening a second socket of its own. What differs between the two @@ -193,15 +190,14 @@ type ResumeOptions struct { // Resume carries on a conversation held in writing, by the id of the session it was held // in, and starts watching it. One that ended is reopened with what was said in it, the tools -// its caller declared and the tool timeout it ran with. The agent's functions, when it has -// any, are declared in place of those tools; a conversation still running keeps its tools -// until it ends. At most one ResumeOptions is read. +// its caller declared and the tool timeout it ran with; a conversation still running keeps +// its tools until it ends. At most one ResumeOptions is read. func (s *Sessions) Resume(ctx context.Context, id string, options ...ResumeOptions) (*Session, error) { var chosen ResumeOptions if len(options) > 0 { chosen = options[0] } - pipeline := stream.Accelerated(stream.Config{Backend: s.client.backend, Functions: s.agent.tools}) + pipeline := stream.Accelerated(stream.Config{Backend: s.client.backend}) got, err := pipeline.Resume(ctx, id, chosen.ToolTimeout) if err != nil { return nil, err @@ -562,6 +558,17 @@ func (s *Session) RemoveTool(ctx context.Context, name string) error { // Events yields what the backend did until the conversation ends, when the channel closes. func (s *Session) Events() <-chan stream.Event { return s.pipeline.Events() } +// On calls handle with every event of kind until off is called. See stream.Pipeline.On. +func (s *Session) On(kind string, handle func(stream.Event)) (off func()) { + return s.pipeline.On(kind, handle) +} + +// ToolCall is the last state the conversation reported for a tool call. See +// stream.Pipeline.ToolCall. +func (s *Session) ToolCall(id string) (acceleration.ToolCallState, bool) { + return s.pipeline.ToolCall(id) +} + // Say speaks text without going through the model, for when the words were never in question. func (s *Session) Say(text string, interrupt bool) error { return s.pipeline.Say(text, interrupt) } diff --git a/sdks/go/examples/caregiving/main.go b/sdks/go/examples/caregiving/main.go index e799237042..e256f6156c 100644 --- a/sdks/go/examples/caregiving/main.go +++ b/sdks/go/examples/caregiving/main.go @@ -116,8 +116,6 @@ type bookings struct { func (b *bookings) book(ctx context.Context, in BookInterview) (any, error) { toolCallContext := tools.ToolCallContextFrom(ctx) - toolCallContext.Update(ctx, "Checking the caregiver's calendar") - b.mu.Lock() defer b.mu.Unlock() if confirmation, ok := b.byKey[toolCallContext.ID]; ok { diff --git a/sdks/go/examples/quickstart/agents/myagent/agent.yaml b/sdks/go/examples/quickstart/agents/myagent/agent.yaml index 41149cd1bd..a2eff6bc16 100644 --- a/sdks/go/examples/quickstart/agents/myagent/agent.yaml +++ b/sdks/go/examples/quickstart/agents/myagent/agent.yaml @@ -1,2 +1,5 @@ name: myagent description: A support agent that answers questions about orders. +tools: + - name: orders + tools: [lookup_order] diff --git a/sdks/go/examples/text/agents/jean/agent.yaml b/sdks/go/examples/text/agents/jean/agent.yaml new file mode 100644 index 0000000000..9e0bb2c5c0 --- /dev/null +++ b/sdks/go/examples/text/agents/jean/agent.yaml @@ -0,0 +1,4 @@ +name: jean +tools: + - name: weather + tools: [get_weather] diff --git a/sdks/go/examples/voice/agents/jean/agent.yaml b/sdks/go/examples/voice/agents/jean/agent.yaml new file mode 100644 index 0000000000..9e0bb2c5c0 --- /dev/null +++ b/sdks/go/examples/voice/agents/jean/agent.yaml @@ -0,0 +1,4 @@ +name: jean +tools: + - name: weather + tools: [get_weather] diff --git a/sdks/go/stream/accelerated.go b/sdks/go/stream/accelerated.go index d4d1238630..c8b703645f 100644 --- a/sdks/go/stream/accelerated.go +++ b/sdks/go/stream/accelerated.go @@ -167,9 +167,11 @@ type Participant struct { // Event is one thing the conversation did. // // Kind is the backend's own name for it: joined, heard, responding, response_delta, -// reasoning_delta (the model's thinking on a written reply), responded, spoke, turn, delegated, task_settled, task_cancelled, tool_ran, transferred, -// pressed, looked_up, backchannel, interrupted, overlap_decided, conversation_compacted, -// error and left. The fields below are filled from whichever of those carry them, and Frame +// reasoning_delta (the model's thinking on a written reply), responded, spoke, turn, +// delegated, task_settled, task_cancelled, tool_started, tool_updated, tool_ran, +// tool_call.started, tool_call.progress, tool_call.finished, transferred, pressed, +// looked_up, backchannel, interrupted, overlap_decided, conversation_compacted, error and +// left. The fields below are filled from whichever of those carry them, and Frame // is the whole thing for anything they do not cover. type Event struct { Kind string @@ -184,7 +186,10 @@ type Event struct { // Resumed is set on a resumed event, which a text session's socket is sent after it // dropped and reconnected. Resumed *Resumption - Frame Frame + // ToolCall is a tool_call.* event's state of the call: the whole of it, so the last one + // per ToolCallId is where the call stands (Pipeline.ToolCall). + ToolCall *acceleration.ToolCallState + Frame Frame } // Resumption is what a socket that reconnected was told. @@ -235,6 +240,7 @@ type Pipeline struct { running sync.WaitGroup watcher sync.WaitGroup holds holds + listeners listeners // changing runs one change of the session's tools at a time, across its request. changing sync.Mutex @@ -1005,8 +1011,10 @@ func (p *Pipeline) read(ctx context.Context, socket *Socket, out chan<- Event, c } } + event := eventOf(frame) + p.observe(event) select { - case out <- eventOf(frame): + case out <- event: case <-ctx.Done(): return ctx.Err() } @@ -1192,6 +1200,7 @@ func eventOf(frame Frame) Event { Error: frame.String("error"), Frame: frame, } + event.ToolCall, _ = toolCallOf(frame) files, _ := frame["files"].([]any) for _, listed := range files { file, ok := listed.(map[string]any) diff --git a/sdks/go/stream/dispatch.go b/sdks/go/stream/dispatch.go index f46172baf6..e487605229 100644 --- a/sdks/go/stream/dispatch.go +++ b/sdks/go/stream/dispatch.go @@ -215,33 +215,29 @@ func (d *Dispatch) OnCall(handler CallHandler) { d.call = handler } // OnMessage registers what to do with a message written to an agent that is not running. func (d *Dispatch) OnMessage(handler MessageHandler) { d.message = handler } -// hosting is one set of functions this worker runs for every session under an agent id. +// hosting is one set of functions this worker runs for every agent of the app. type hosting struct { - agentID string functions *tools.Registry toolTimeout time.Duration } -// HostedAgent is an agent whose tools a worker can host: a client.Agent or an agents.Agent. +// ToolHost holds the tools a worker can host: a client.Client. // -// An interface rather than either, because both of those are built on this package and it -// cannot name them. -type HostedAgent interface { - Name() string +// An interface rather than the client, because it is built on this package and this +// package cannot name it. +type ToolHost interface { Tools() *tools.Registry } -// Host runs an agent's tools for every session opened under it, whoever opened it. +// Host runs the app's tools for every session that is offered them, whoever opened it. // -// A session's own functions run in the process that opened it, which is no use to a -// conversation opened from a browser that wants to read a source tree. Hosting is the other -// direction: the router offers the agent's tools to each session naming the agent and sends -// every call to one of them here. The router matches a session on its agent id or its -// agent's name, so the name is what the agent is hosted under. toolTimeout is how long the -// router waits for one tool call to be answered before telling the model it failed, not how -// long the worker runs; zero takes the router's default of 30 seconds. Call before Run. -func (d *Dispatch) Host(agent HostedAgent, toolTimeout time.Duration) { - d.hosted = append(d.hosted, hosting{agentID: agent.Name(), functions: agent.Tools(), toolTimeout: toolTimeout}) +// The tools are registered once for every agent: the router offers one to a session only +// when a tool group of its agent's agent.yaml names it, and sends every call of it to one of +// the workers hosting it. toolTimeout is how long the router waits for one tool call to be +// answered before telling the model it failed, not how long the worker runs; zero takes the +// router's default of 30 seconds. Call before Run. +func (d *Dispatch) Host(host ToolHost, toolTimeout time.Duration) { + d.hosted = append(d.hosted, hosting{functions: host.Tools(), toolTimeout: toolTimeout}) } // host tells the router what this worker runs, once it is listening. @@ -261,7 +257,7 @@ func (d *Dispatch) host() { } declared = append(declared, tool) } - d.tell(Frame{"type": "host_tools", "agent_id": offer.agentID, "tools": declared, "timeout_ms": offer.toolTimeout.Milliseconds()}) + d.tell(Frame{"type": "host_tools", "every_agent": true, "tools": declared, "timeout_ms": offer.toolTimeout.Milliseconds()}) } } @@ -499,12 +495,11 @@ func (d *Dispatch) read(ctx context.Context, socket *Socket) error { offer.functions.Cancel(frame.String("session_id"), frame.String("id")) } case "hosting": - d.logger.Info("the router sends this worker's tools here", "agent", frame.String("agent_id")) + d.logger.Info("the router sends this worker's tools here", "tools", frame["tools"]) case "hosting_refused": // Not worth waiting on: a worker whose tools were refused is one nobody will // call, and saying so is better than sitting connected looking healthy. - return fmt.Errorf("%w for agent %s: %s", errRefused, - frame.String("agent_id"), frame.String("reason")) + return fmt.Errorf("%w: %s", errRefused, frame.String("reason")) case "pong": select { case d.pong <- struct{}{}: diff --git a/sdks/go/stream/dispatch_test.go b/sdks/go/stream/dispatch_test.go index a9c0eeaf3f..dbf85ddb2e 100644 --- a/sdks/go/stream/dispatch_test.go +++ b/sdks/go/stream/dispatch_test.go @@ -102,14 +102,11 @@ func waiting(t *testing.T, router *pool, options DispatchOptions) *Dispatch { return worker } -// hostedAgent is an agent to host, standing in for a client.Agent this package cannot import. -type hostedAgent struct { - name string - functions *tools.Registry -} +// toolHost is the app's tools to host, standing in for a client.Client this package cannot +// import. +type toolHost struct{ functions *tools.Registry } -func (a hostedAgent) Name() string { return a.name } -func (a hostedAgent) Tools() *tools.Registry { return a.functions } +func (h toolHost) Tools() *tools.Registry { return h.functions } // run waits for work in the background and returns what Run said once it stops. func run(t *testing.T, ctx context.Context, worker *Dispatch) func() error { @@ -429,7 +426,7 @@ func TestAHostedToolCallIsNotCountedAsWorkTheRouterHandedOut(t *testing.T) { t.Fatal(err) } worker := waiting(t, router, DispatchOptions{}) - worker.Host(hostedAgent{"stream-support", functions}, time.Minute) + worker.Host(toolHost{functions}, time.Minute) ctx, stop := context.WithCancel(t.Context()) defer stop() @@ -543,13 +540,13 @@ func TestAHostedFunctionIsDeclaredAndAnsweredOverTheDispatchSocket(t *testing.T) t.Fatal(err) } worker := waiting(t, router, DispatchOptions{}) - worker.Host(hostedAgent{"stream-support", functions}, time.Minute) + worker.Host(toolHost{functions}, time.Minute) ctx, cancel := context.WithCancel(t.Context()) stopped := run(t, ctx, worker) declared := router.told(t, "host_tools") - if declared.String("agent_id") != "stream-support" || declared.Int("timeout_ms") != 60000 { + if declared["every_agent"] != true || declared["agent_id"] != nil || declared.Int("timeout_ms") != 60000 { t.Errorf("the router was told %v", declared) } answered := router.told(t, "tool_result") @@ -585,7 +582,7 @@ func TestAWorkerTheRouterDropsReconnectsAndHostsAgain(t *testing.T) { } worker := waiting(t, router, DispatchOptions{}) worker.firstRetry = 10 * time.Millisecond - worker.Host(hostedAgent{"stream-support", functions}, 0) + worker.Host(toolHost{functions}, 0) ctx, cancel := context.WithCancel(t.Context()) stopped := run(t, ctx, worker) @@ -604,7 +601,7 @@ func TestAWorkerTheRouterDropsReconnectsAndHostsAgain(t *testing.T) { case <-time.After(10 * time.Millisecond): } } - if declared := router.told(t, "host_tools"); declared.String("agent_id") != "stream-support" { + if declared := router.told(t, "host_tools"); declared["every_agent"] != true { t.Errorf("the reconnected worker declared %v", declared) } cancel() @@ -615,14 +612,14 @@ func TestAWorkerTheRouterDropsReconnectsAndHostsAgain(t *testing.T) { func TestAWorkerWhoseToolsAreRefusedStopsWaiting(t *testing.T) { router := newPool(t, func(connection *websocket.Conn) { - _ = connection.WriteJSON(Frame{"type": "hosting_refused", "agent_id": "stream-support", "reason": "hosting no tools is not hosting"}) + _ = connection.WriteJSON(Frame{"type": "hosting_refused", "reason": "hosting no tools is not hosting"}) }) functions := tools.NewRegistry() if err := tools.Register(functions, "investigate_sdk", "Read SDK source", func(context.Context, struct{}) (any, error) { return "", nil }); err != nil { t.Fatal(err) } worker := waiting(t, router, DispatchOptions{}) - worker.Host(hostedAgent{"stream-support", functions}, 0) + worker.Host(toolHost{functions}, 0) if err := run(t, t.Context(), worker)(); err == nil { t.Fatal("a worker nobody will call kept waiting") diff --git a/sdks/go/stream/hosted_test.go b/sdks/go/stream/hosted_test.go index fc65c4287a..4d0eb9e3bf 100644 --- a/sdks/go/stream/hosted_test.go +++ b/sdks/go/stream/hosted_test.go @@ -28,7 +28,7 @@ func (s *HostedCallSuite) answer(photograph func(context.Context, struct{}) (any functions := tools.NewRegistry() s.Require().NoError(tools.Register(functions, "photograph_room", "Photograph the room", photograph)) worker := waiting(s.T(), router, DispatchOptions{}) - worker.Host(hostedAgent{"stream-support", functions}, time.Minute) + worker.Host(toolHost{functions}, time.Minute) ctx, cancel := context.WithCancel(s.T().Context()) stopped := run(s.T(), ctx, worker) s.T().Cleanup(func() { @@ -46,7 +46,7 @@ func (s *HostedCallSuite) TestAWorkerDeclaresEachToolsPolicies() { tools.Interruptible(true), tools.Timeout(90*time.Second), tools.NeedsApproval("Read the SDK source?"))) s.Require().NoError(tools.Register(functions, "search_docs", "Search the docs", noop)) worker := waiting(s.T(), router, DispatchOptions{}) - worker.Host(hostedAgent{"stream-support", functions}, time.Minute) + worker.Host(toolHost{functions}, time.Minute) ctx, cancel := context.WithCancel(s.T().Context()) stopped := run(s.T(), ctx, worker) defer func() { @@ -69,6 +69,36 @@ func (s *HostedCallSuite) TestAWorkerDeclaresEachToolsPolicies() { s.NotContains(byName["search_docs"], "approval") } +func (s *HostedCallSuite) TestTheAppsToolsAreHostedOnceForEveryAgentAndAnswerAnySession() { + router := newPool(s.T(), func(connection *websocket.Conn) { + _ = connection.WriteJSON(Frame{"type": "tool_call", "id": "call-1", "session_id": "sales-session", "name": "lookup_order", "arguments": "{}"}) + _ = connection.WriteJSON(Frame{"type": "tool_call", "id": "call-2", "session_id": "support-session", "name": "lookup_order", "arguments": "{}"}) + }) + functions := tools.NewRegistry() + s.Require().NoError(tools.Register(functions, "lookup_order", "Look up an order", func(ctx context.Context, _ struct{}) (any, error) { + return "shipped for " + tools.ToolCallContextFrom(ctx).SessionID, nil + })) + worker := waiting(s.T(), router, DispatchOptions{}) + worker.Host(toolHost{functions}, time.Minute) + ctx, cancel := context.WithCancel(s.T().Context()) + stopped := run(s.T(), ctx, worker) + defer func() { + cancel() + stopped() + }() + + declared := router.told(s.T(), "host_tools") + s.Equal(true, declared["every_agent"]) + s.NotContains(declared, "agent_id") + answers := map[string]string{} + for range 2 { + answered := router.told(s.T(), "tool_result") + answers[answered.String("tool_call_id")] = answered.String("output") + } + + s.Equal(map[string]string{"call-1": "shipped for sales-session", "call-2": "shipped for support-session"}, answers) +} + func (s *HostedCallSuite) TestAFunctionReadsTheCallItIsRunningFor() { router := newPool(s.T(), func(connection *websocket.Conn) { _ = connection.WriteJSON(Frame{ @@ -84,7 +114,7 @@ func (s *HostedCallSuite) TestAFunctionReadsTheCallItIsRunningFor() { return "the kitchen", nil })) worker := waiting(s.T(), router, DispatchOptions{}) - worker.Host(hostedAgent{"stream-support", functions}, time.Minute) + worker.Host(toolHost{functions}, time.Minute) ctx, cancel := context.WithCancel(s.T().Context()) stopped := run(s.T(), ctx, worker) defer func() { @@ -124,7 +154,7 @@ func (s *HostedCallSuite) TestAPictureReachesTheRouter() { s.Equal("https://example.com/kitchen.jpg", parts[1].(map[string]any)["image_url"].(map[string]any)["url"]) } -func (s *HostedCallSuite) TestAHostedFunctionSaysHowItIsGoingAndHoldsItsWrite() { +func (s *HostedCallSuite) TestAHostedFunctionHoldsItsWrite() { heard := make(chan Frame, 32) router := newPool(s.T(), func(connection *websocket.Conn) { if connection.WriteJSON(Frame{"type": "tool_call", "id": "call-1", "session_id": "s", "name": "rebook_flight", "arguments": "{}"}) != nil { @@ -144,18 +174,14 @@ func (s *HostedCallSuite) TestAHostedFunctionSaysHowItIsGoingAndHoldsItsWrite() }) functions := tools.NewRegistry() s.Require().NoError(tools.Register(functions, "rebook_flight", "Move a booking", func(ctx context.Context, _ struct{}) (any, error) { - toolCallContext := tools.ToolCallContextFrom(ctx) - if err := toolCallContext.Update(ctx, "Checking seats"); err != nil { - return nil, err - } - writing := toolCallContext.DisallowInterruptions(ctx) + writing := tools.ToolCallContextFrom(ctx).DisallowInterruptions(ctx) if writing.Err() != nil { return nil, writing.Err() } return "rebooked", nil })) worker := waiting(s.T(), router, DispatchOptions{}) - worker.Host(hostedAgent{"stream-support", functions}, time.Minute) + worker.Host(toolHost{functions}, time.Minute) ctx, cancel := context.WithCancel(s.T().Context()) stopped := run(s.T(), ctx, worker) defer func() { @@ -170,7 +196,6 @@ func (s *HostedCallSuite) TestAHostedFunctionSaysHowItIsGoingAndHoldsItsWrite() told = append(told, frame) } } - s.Require().Len(told, 2) - s.Equal(Frame{"type": "tool_update", "tool_call_id": "call-1", "text": "Checking seats", "verbatim": false}, told[0]) - s.Equal(Frame{"type": "tool_hold", "tool_call_id": "call-1"}, told[1]) + s.Require().Len(told, 1) + s.Equal(Frame{"type": "tool_hold", "tool_call_id": "call-1"}, told[0]) } diff --git a/sdks/go/stream/tool_calls.go b/sdks/go/stream/tool_calls.go new file mode 100644 index 0000000000..52944e1d10 --- /dev/null +++ b/sdks/go/stream/tool_calls.go @@ -0,0 +1,96 @@ +package stream + +import ( + "encoding/json" + "strings" + "sync" + + "github.com/GetStream/Vision-Agents/sdks/go/acceleration" +) + +// listeners are the handlers On registered, by the kind of event each wants, and the last +// state of every tool call the session reported. +type listeners struct { + mu sync.Mutex + next int + by map[string]map[int]func(Event) + toolCalls map[string]acceleration.ToolCallState +} + +// On calls handle with every event of kind the session reports, such as "tool_call.finished" +// or "responded", until off is called. Handlers run one at a time on the goroutine reading +// the session, before the event reaches Events, so a slow one holds up the rest. +func (p *Pipeline) On(kind string, handle func(Event)) (off func()) { + p.listeners.mu.Lock() + defer p.listeners.mu.Unlock() + if p.listeners.by == nil { + p.listeners.by = map[string]map[int]func(Event){} + } + if p.listeners.by[kind] == nil { + p.listeners.by[kind] = map[int]func(Event){} + } + id := p.listeners.next + p.listeners.next++ + p.listeners.by[kind][id] = handle + return func() { + p.listeners.mu.Lock() + defer p.listeners.mu.Unlock() + delete(p.listeners.by[kind], id) + } +} + +// ToolCall is the last state the session reported for the tool call id, whichever process +// runs it, reporting false for a call it has reported nothing about. +func (p *Pipeline) ToolCall(id string) (acceleration.ToolCallState, bool) { + p.listeners.mu.Lock() + defer p.listeners.mu.Unlock() + state, ok := p.listeners.toolCalls[id] + return state, ok +} + +// ToolCalls is the last state of every tool call the session reported, finished ones too. +func (p *Pipeline) ToolCalls() []acceleration.ToolCallState { + p.listeners.mu.Lock() + defer p.listeners.mu.Unlock() + states := make([]acceleration.ToolCallState, 0, len(p.listeners.toolCalls)) + for _, state := range p.listeners.toolCalls { + states = append(states, state) + } + return states +} + +// observe keeps the state event carries and hands event to its kind's handlers. +func (p *Pipeline) observe(event Event) { + p.listeners.mu.Lock() + if event.ToolCall != nil { + if p.listeners.toolCalls == nil { + p.listeners.toolCalls = map[string]acceleration.ToolCallState{} + } + p.listeners.toolCalls[event.ToolCall.ToolCallId] = *event.ToolCall + } + handlers := make([]func(Event), 0, len(p.listeners.by[event.Kind])) + for _, handle := range p.listeners.by[event.Kind] { + handlers = append(handlers, handle) + } + p.listeners.mu.Unlock() + for _, handle := range handlers { + handle(event) + } +} + +// toolCallOf reads a tool_call.started, tool_call.progress or tool_call.finished frame, +// reporting false for any other. +func toolCallOf(frame Frame) (*acceleration.ToolCallState, bool) { + if !strings.HasPrefix(frame.Type(), "tool_call.") { + return nil, false + } + raw, err := json.Marshal(frame) + if err != nil { + return nil, false + } + var state acceleration.ToolCallState + if json.Unmarshal(raw, &state) != nil || state.ToolCallId == "" { + return nil, false + } + return &state, true +} diff --git a/sdks/go/stream/tool_calls_test.go b/sdks/go/stream/tool_calls_test.go new file mode 100644 index 0000000000..cea36de3fb --- /dev/null +++ b/sdks/go/stream/tool_calls_test.go @@ -0,0 +1,85 @@ +package stream + +import ( + "testing" + + "github.com/gorilla/websocket" + "github.com/stretchr/testify/suite" + + "github.com/GetStream/Vision-Agents/sdks/go/acceleration" +) + +// ToolCallStateSuite is how a session's tool call states reach the program watching it. +type ToolCallStateSuite struct { + suite.Suite +} + +func TestToolCallStateSuite(t *testing.T) { + suite.Run(t, new(ToolCallStateSuite)) +} + +// watch opens a pipeline on a router that sends frames and ends the session, calling before +// on it first so nothing is missed, and returns it with what reached Events. +func (s *ToolCallStateSuite) watch(before func(*Pipeline), frames ...Frame) (*Pipeline, []Event) { + router := newRestartable(s.T(), func(connection *websocket.Conn) { + for _, frame := range frames { + _ = connection.WriteJSON(frame) + } + closes(connection, websocket.CloseNormalClosure, "the session ended") + }) + pipeline := Accelerated(Config{Backend: Backend{URL: router.url(), CustomerID: "acme"}}) + before(pipeline) + s.Require().NoError(pipeline.Watch(s.T().Context(), &acceleration.Session{Id: "session-1"})) + return pipeline, collect(s.T(), pipeline.Events()) +} + +func (s *ToolCallStateSuite) TestTheLastStateOfEachCallIsKept() { + pipeline, events := s.watch(func(*Pipeline) {}, + Frame{"type": "tool_call.started", "tool_call_id": "call-1", "tool": "book_flight", "turn_id": "turn-1", + "status": "waiting", "started_at": "2026-11-18T12:00:00Z", "duration_ms": 0}, + Frame{"type": "tool_call.started", "tool_call_id": "call-2", "tool": "search_flights", "turn_id": "turn-1", + "status": "in_progress", "started_at": "2026-11-18T12:00:00Z", "duration_ms": 0}, + Frame{"type": "tool_call.progress", "tool_call_id": "call-1", "tool": "book_flight", "turn_id": "turn-1", + "status": "in_progress", "started_at": "2026-11-18T12:00:00Z", "duration_ms": 400, + "approved_at": "2026-11-18T12:00:00.4Z"}, + Frame{"type": "tool_call.finished", "tool_call_id": "call-1", "tool": "book_flight", "turn_id": "turn-1", + "status": "success", "started_at": "2026-11-18T12:00:00Z", "duration_ms": 1200, + "approved_at": "2026-11-18T12:00:00.4Z"}, + ) + + booked, ok := pipeline.ToolCall("call-1") + s.Require().True(ok) + s.Equal(acceleration.ToolCallStatus("success"), booked.Status) + s.EqualValues(1200, booked.DurationMs) + s.Require().NotNil(booked.ApprovedAt) + searching, ok := pipeline.ToolCall("call-2") + s.Require().True(ok) + s.Equal(acceleration.ToolCallStatus("in_progress"), searching.Status) + s.Len(pipeline.ToolCalls(), 2) + s.Require().Len(events, 4) + s.Require().NotNil(events[3].ToolCall) + s.Equal("call-1", events[3].ToolCall.ToolCallId) +} + +func (s *ToolCallStateSuite) TestAHandlerHearsItsKindUntilItIsTurnedOff() { + var heard []string + var finished int + s.watch(func(pipeline *Pipeline) { + var off func() + off = pipeline.On("tool_call.progress", func(event Event) { + heard = append(heard, *event.ToolCall.Progress) + off() + }) + pipeline.On("tool_call.finished", func(Event) { finished++ }) + }, + Frame{"type": "tool_call.progress", "tool_call_id": "call-1", "tool": "search_flights", "status": "in_progress", + "started_at": "2026-11-18T12:00:00Z", "duration_ms": 100, "progress": "Searching for flights"}, + Frame{"type": "tool_call.progress", "tool_call_id": "call-1", "tool": "search_flights", "status": "in_progress", + "started_at": "2026-11-18T12:00:00Z", "duration_ms": 200, "progress": "Comparing prices"}, + Frame{"type": "tool_call.finished", "tool_call_id": "call-1", "tool": "search_flights", "status": "success", + "started_at": "2026-11-18T12:00:00Z", "duration_ms": 300}, + ) + + s.Equal([]string{"Searching for flights"}, heard) + s.Equal(1, finished) +} diff --git a/sdks/go/stream/tool_line.go b/sdks/go/stream/tool_line.go index 1e59371084..47aeafd2b7 100644 --- a/sdks/go/stream/tool_line.go +++ b/sdks/go/stream/tool_line.go @@ -17,10 +17,6 @@ type toolLine struct { holds *holds } -func (l toolLine) Update(_ context.Context, id, text string, verbatim bool) error { - return l.send(Frame{"type": "tool_update", "tool_call_id": id, "text": text, "verbatim": verbatim}) -} - func (l toolLine) Hold(ctx context.Context, id string) (bool, error) { return l.holds.ask(ctx, id, func() error { return l.send(Frame{"type": "tool_hold", "tool_call_id": id}) }) } diff --git a/sdks/go/stream/tool_progress_test.go b/sdks/go/stream/tool_progress_test.go index af7eb5468b..75871f9aba 100644 --- a/sdks/go/stream/tool_progress_test.go +++ b/sdks/go/stream/tool_progress_test.go @@ -23,42 +23,6 @@ func toldTheRouter(connection *websocket.Conn, n int) []map[string]any { return told } -func (s *ToolCallSuite) TestAFunctionSaysHowItIsGoing() { - heard := make(chan []map[string]any, 1) - backend := newRouter(s.T(), func(t *testing.T, connection *websocket.Conn) { - if connection.WriteJSON(map[string]any{ - "type": "tool_call", "id": "call-1", "name": "lookup_order", "arguments": "{}", - }) != nil { - return - } - heard <- toldTheRouter(connection, 3) - }) - pipeline := Accelerated(Config{Backend: Backend{URL: backend.URL, CustomerID: "acme"}}) - s.Require().NoError(tools.Register(pipeline.Functions(), "lookup_order", "Look an order up", - func(ctx context.Context, _ struct{}) (any, error) { - toolCallContext := tools.ToolCallContextFrom(ctx) - if err := toolCallContext.Update(ctx, "Looking up order 7"); err != nil { - return nil, err - } - if err := toolCallContext.Say(ctx, "One moment."); err != nil { - return nil, err - } - return "shipped", nil - })) - - s.join(pipeline) - - select { - case told := <-heard: - s.Require().Len(told, 3) - s.Equal(map[string]any{"type": "tool_update", "tool_call_id": "call-1", "text": "Looking up order 7", "verbatim": false}, told[0]) - s.Equal(map[string]any{"type": "tool_update", "tool_call_id": "call-1", "text": "One moment.", "verbatim": true}, told[1]) - s.Equal("tool_result", told[2]["type"]) - case <-time.After(3 * time.Second): - s.Fail("the router never heard how the call was going") - } -} - func (s *ToolCallSuite) TestAHeldCallOutlivesTheRouterCancellingIt() { answered := make(chan map[string]any, 1) backend := newRouter(s.T(), func(t *testing.T, connection *websocket.Conn) { diff --git a/sdks/go/tools/call.go b/sdks/go/tools/call.go index 4af87433c8..754ddaffd2 100644 --- a/sdks/go/tools/call.go +++ b/sdks/go/tools/call.go @@ -25,11 +25,9 @@ type Call struct { } // Line is how a running call reaches the session it runs for. The socket that delivered the -// call puts it in the call's context (WithLine); without one, as outside a call, Update, Say -// and DisallowInterruptions do nothing. +// call puts it in the call's context (WithLine); without one, as outside a call, +// DisallowInterruptions does nothing. type Line interface { - // Update says how the call is going, or with verbatim, words to say as written. - Update(ctx context.Context, id, text string, verbatim bool) error // Hold asks for the call to keep going when the caller talks over the agent, and // reports whether it was still running to be held. Hold(ctx context.Context, id string) (bool, error) @@ -39,27 +37,6 @@ type Line interface { // had already been stopped. var ErrNotHeld = errors.New("tools: the call was stopped before it could be held") -// Update says how the call is going. The agent tells a voice caller in its own words once -// nobody is talking, and a conversation shows it on the call's step until the call ends. A -// newer update replaces one not yet said, and the result replaces both. -func (c *Call) Update(ctx context.Context, text string) error { - line := lineFrom(ctx) - if line == nil { - return nil - } - return line.Update(ctx, c.ID, text, false) -} - -// Say has the agent say text as written while the call runs, or post it in a text -// conversation. A speech-to-speech model is asked for these words and may paraphrase them. -func (c *Call) Say(ctx context.Context, text string) error { - line := lineFrom(ctx) - if line == nil { - return nil - } - return line.Update(ctx, c.ID, text, true) -} - // DisallowInterruptions keeps the call going from now on when the caller talks over the // agent, which is what a write that must not be left halfway wants. It returns the context // to do that write on: only the call's timeout and its socket ending cancel it, and not the diff --git a/sdks/go/tools/tools_test.go b/sdks/go/tools/tools_test.go index 1f0a525531..2ef9355329 100644 --- a/sdks/go/tools/tools_test.go +++ b/sdks/go/tools/tools_test.go @@ -314,16 +314,10 @@ func TestWordsAloneStayAString(t *testing.T) { } } -func TestOutsideACallItsProgressGoesNowhere(t *testing.T) { +func TestOutsideACallHoldingItChangesNothing(t *testing.T) { ctx := context.Background() toolCallContext := ToolCallContextFrom(ctx) - if err := toolCallContext.Update(ctx, "working"); err != nil { - t.Errorf("an update outside a toolCallContext failed: %v", err) - } - if err := toolCallContext.Say(ctx, "one moment"); err != nil { - t.Errorf("words outside a toolCallContext failed: %v", err) - } if held := toolCallContext.DisallowInterruptions(ctx); held != ctx { t.Error("holding a call that is not one changed its context") } diff --git a/sdks/js/README.md b/sdks/js/README.md index 637842c2fc..c04b538c6f 100644 --- a/sdks/js/README.md +++ b/sdks/js/README.md @@ -353,7 +353,6 @@ agent.tools.register<{ flight_id: string }>({ approval: { title: "Book flight {flight_id}?" }, timeoutMs: 20_000, run: async ({ flight_id }, toolCallContext) => { - toolCallContext.update("checking seats"); await toolCallContext.disallowInterruptions(); return bookings.book(flight_id, { user: toolCallContext.user.id, key: toolCallContext.id }); }, @@ -366,7 +365,7 @@ agent.tools.register<{ flight_id: string }>({ longer fails. - `concurrency` caps how many calls to the tool run at once. A call over the cap waits its turn, and its timeout keeps counting while it waits. Every tool's calls together are capped - at 16 at once, which `agent.tools.setConcurrency(n)` changes. A `Dispatch` shares an agent's + at 16 at once, which `agent.tools.setConcurrency(n)` changes. A `Dispatch` shares the client's `tools` across every session it hosts, so the cap is the worker's. - `approval` is asked before each call runs. In writing the call waits in `session.approvals` until `session.decide(id, { allowed })` answers it; a page follows the list in @@ -463,23 +462,24 @@ written to and a `requestId`, unanswered, and `dispatch.answer(message)` has the it, with this worker's credential acting for the user who wrote it. The photos sent with a message come in `images`, by URL, and `dispatch.answer` shows them to the model. -A worker can also host tools for every session opened under an agent id, including one a -browser opened, where the session's own process cannot reach what the tool needs: +A worker can also host the app's tools, registered once on the client for every agent, so they +run for any session, including one a browser opened, where the session's own process cannot +reach what the tool needs: ```ts -const agent = client.agent("my-agent"); -agent.tools.register({ name: "lookup", description: "Look up an order", run: lookup }); -dispatch.host(agent, { toolTimeoutMs: 60_000 }); +const client = new Client(); +client.tools.register({ name: "lookup", description: "Look up an order", timeoutMs: 60_000, run: lookup }); +await client.runDispatch(); ``` -The router offers them to each session naming the agent and sends every call here. A hosted -tool takes the same `interruptible`, `timeoutMs` and `approval` (asked only on a voice -call), is handed its `toolCallContext`, and fails when the router gives up on it. `run` -works with only hosted tools and no handler, and throws `HostingRefusedError` if the router -refuses them. +The router offers a tool to a session only where a tool group of its agent's agent.yaml names +it, and sends every call here. A hosted tool takes the same `interruptible`, `timeoutMs` and +`approval` (asked only on a voice call), is handed its `toolCallContext`, and fails when the +router gives up on it. `runDispatch` rejects with `HostingRefusedError` if the router refuses +them. A worker that also answers calls or messages hosts them with `dispatch.host(client)` +before `run`. -Tool names must be unique across the agents hosted by one worker. Use separate workers when -agents need different implementations of a tool with the same name. `stop` also cancels a +Tool names must be unique across the clients hosted by one worker. `stop` also cancels a socket upgrade still in progress; work already running is drained. Server side only. A worker is offered other people's callers, so anything that can open diff --git a/sdks/js/package-lock.json b/sdks/js/package-lock.json index 6b788db580..a5f9f47aaf 100644 --- a/sdks/js/package-lock.json +++ b/sdks/js/package-lock.json @@ -12,8 +12,12 @@ "@stream-io/node-sdk": "^0.8.15", "@stream-io/video-client": "^1.61.2", "@types/node": "^22.10.0", + "@types/react": "^19.3.0", + "@types/react-dom": "^19.3.0", "@types/ws": "^8.5.13", "openapi-typescript": "^7.13.0", + "react": "^19.3.1", + "react-dom": "^19.3.1", "stream-chat": "^9.54.0", "typescript": "^5.7.0", "ws": "^8.18.0", @@ -218,6 +222,27 @@ "undici-types": "~6.21.0" } }, + "node_modules/@types/react": { + "version": "19.3.0", + "resolved": "https://registry.npmjs.org/@types/react/-/react-19.3.0.tgz", + "integrity": "sha512-N0rFCuH9YoxG9/m61l9MfpJKfmLOVU0em7ipIz6TRgSSkvReLB9vL85GB+yr8Bs5leqpvg96JSwF4ZS1s4viQg==", + "dev": true, + "license": "MIT", + "peer": true, + "dependencies": { + "csstype": "^3.2.2" + } + }, + "node_modules/@types/react-dom": { + "version": "19.3.0", + "resolved": "https://registry.npmjs.org/@types/react-dom/-/react-dom-19.3.0.tgz", + "integrity": "sha512-ZI7bU42mZXXKHn/qNLEw2IrbiINU7X5+vfgdixBHkCNpYWXjKgfQ/P+uyGb5CjOLB9UcnTeg3rylQtV2hym44Q==", + "dev": true, + "license": "MIT", + "peerDependencies": { + "@types/react": "^19.3.0" + } + }, "node_modules/@types/ws": { "version": "8.18.1", "resolved": "https://registry.npmjs.org/@types/ws/-/ws-8.18.1.tgz", @@ -388,6 +413,13 @@ "node": ">= 0.8" } }, + "node_modules/csstype": { + "version": "3.2.4", + "resolved": "https://registry.npmjs.org/csstype/-/csstype-3.2.4.tgz", + "integrity": "sha512-PoKaoPzQapnEoqUSnHx1u6W6ku8CVEgKsnpmmzU3m1nwkidpPBQ8V/Zq2+bWNrsOKoKLwTGdbx6xF7BXP1zLVQ==", + "dev": true, + "license": "MIT" + }, "node_modules/debug": { "version": "4.4.3", "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", @@ -944,6 +976,30 @@ "node": ">=10" } }, + "node_modules/react": { + "version": "19.3.1", + "resolved": "https://registry.npmjs.org/react/-/react-19.3.1.tgz", + "integrity": "sha512-QNhWdt7nrgVWWQG58QuGsAR3jv3Hbe4mb4vz7w2csnHavciZJgda1CPNaldF6eVxv90ud3jUXpKlHL69gO38yw==", + "dev": true, + "license": "MIT", + "peer": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/react-dom": { + "version": "19.3.1", + "resolved": "https://registry.npmjs.org/react-dom/-/react-dom-19.3.1.tgz", + "integrity": "sha512-Y0bARz53lhEC1NMCASs0/j+faB/zqt3R8hpvg1e8+61ZVJ4UL0UdC4cBbXfZMcIuhrfgeILTLe+ZeBCavQPtKQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "scheduler": "^0.28.0" + }, + "peerDependencies": { + "react": "^19.3.1" + } + }, "node_modules/require-from-string": { "version": "2.0.2", "resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz", @@ -985,6 +1041,13 @@ ], "license": "MIT" }, + "node_modules/scheduler": { + "version": "0.28.0", + "resolved": "https://registry.npmjs.org/scheduler/-/scheduler-0.28.0.tgz", + "integrity": "sha512-juorfCmIkIw8tT+p5BXSm6PJjQF/ycEYmKyzURCIt/RaZIhL+PulbQ9Yu2z1HdOJDdqDTlxA1+xKBmHXJsczAw==", + "dev": true, + "license": "MIT" + }, "node_modules/sdp": { "version": "3.2.2", "resolved": "https://registry.npmjs.org/sdp/-/sdp-3.2.2.tgz", @@ -1076,6 +1139,7 @@ "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", "dev": true, "license": "Apache-2.0", + "peer": true, "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" @@ -1145,6 +1209,7 @@ "integrity": "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw==", "dev": true, "license": "MIT", + "peer": true, "engines": { "node": ">=10.0.0" }, diff --git a/sdks/js/package.json b/sdks/js/package.json index 35a4505e14..3ae72ae13e 100644 --- a/sdks/js/package.json +++ b/sdks/js/package.json @@ -42,6 +42,10 @@ "./video": { "types": "./dist/video.d.ts", "default": "./dist/video.js" + }, + "./react": { + "types": "./dist/react.d.ts", + "default": "./dist/react.js" } }, "files": [ @@ -69,8 +73,12 @@ "@stream-io/node-sdk": "^0.8.15", "@stream-io/video-client": "^1.61.2", "@types/node": "^22.10.0", + "@types/react": "^19.3.0", + "@types/react-dom": "^19.3.0", "@types/ws": "^8.5.13", "openapi-typescript": "^7.13.0", + "react": "^19.3.1", + "react-dom": "^19.3.1", "stream-chat": "^9.54.0", "typescript": "^5.7.0", "ws": "^8.18.0", @@ -78,7 +86,8 @@ }, "peerDependencies": { "@stream-io/video-client": "^1.0.0", - "stream-chat": "^9.0.0 || ^10.0.2" + "stream-chat": "^9.0.0 || ^10.0.2", + "react": ">=18" }, "peerDependenciesMeta": { "@stream-io/video-client": { @@ -86,6 +95,9 @@ }, "stream-chat": { "optional": true + }, + "react": { + "optional": true } } } diff --git a/sdks/js/src/client.ts b/sdks/js/src/client.ts index c5f69aedd5..a46d916886 100644 --- a/sdks/js/src/client.ts +++ b/sdks/js/src/client.ts @@ -16,6 +16,7 @@ import { type GuestStore, type GuestUserOptions, } from "./guests.js"; +import { Dispatch } from "./dispatch.js"; import { AgentHandle } from "./handle.js"; import { Memories } from "./memories.js"; import { Router, type RouterOptions } from "./router.js"; @@ -24,6 +25,7 @@ import { Simulations } from "./simulations.js"; import { SpendLimits } from "./spend-limits.js"; import { StreamPeers, type StreamClients } from "./stream.js"; import { Teams } from "./teams.js"; +import { Tools } from "./tools.js"; /** The schemas from the spec, so callers can name a request or a response they build. */ export type Schemas = components["schemas"]; @@ -141,6 +143,11 @@ export class Client { readonly spendLimits: SpendLimits; /** The organization's teams and their spend limits. Server side only. */ readonly teams: Teams; + /** + * The app's tools, registered once for every agent and run by `runDispatch`. An agent is + * offered one only where a tool group of its agent.yaml names it. + */ + readonly tools = new Tools(); /** @internal The chat and video clients this client's sessions share. */ readonly peers: StreamPeers; @@ -248,6 +255,22 @@ export class Client { return new AgentHandle(this, name); } + /** + * Runs the app's tools for every session they are offered to, whoever opened it, until + * `signal` aborts. A tool's own `timeoutMs` is how long the router waits for one of its + * calls; without one it waits 30 seconds. A worker that also answers calls or messages + * hosts the tools with `dispatch.host(client)` instead. + * + * ```ts + * const client = new Client(); + * client.tools.register({ name: "weather_lookup", description: "Look up the weather", run }); + * await client.runDispatch(); + * ``` + */ + runDispatch(signal?: AbortSignal): Promise { + return new Dispatch({ client: this }).host(this).run(signal); + } + /** * Gets or creates a guest, so somebody can talk to an agent before they sign up. * diff --git a/sdks/js/src/dispatch.ts b/sdks/js/src/dispatch.ts index c568491ba1..968cd98d42 100644 --- a/sdks/js/src/dispatch.ts +++ b/sdks/js/src/dispatch.ts @@ -2,7 +2,6 @@ import type { Agent } from "./agent.js"; import type { BackendOptions } from "./backend.js"; import { Client } from "./client.js"; import { ConfigurationError, HostingRefusedError, SocketClosedError } from "./errors.js"; -import type { AgentHandle } from "./handle.js"; import { callOf, messageOf, type InboundCall, type InboundMessage } from "./inbound.js"; import { createResponse, type AgentResponse } from "./responses.js"; import type { Session } from "./session.js"; @@ -64,9 +63,8 @@ export interface HostOptions { toolTimeoutMs?: number; } -/** One set of functions this worker runs for every session under an agent id. */ +/** One set of functions this worker runs for every agent of the app. */ interface Hosting { - agentId: string; tools: Tools; toolTimeoutMs: number; } @@ -174,25 +172,20 @@ export class Dispatch { } /** - * Runs an agent's tools for every session opened under it, whoever opened it. + * Runs the app's tools for every session that is offered them, whoever opened it. * - * A session's own functions run in the process that opened it, which is no use to a - * conversation opened from a browser that wants to read a source tree. Hosting is the - * other direction: the router offers `agent.tools` to each session naming the agent and - * sends every call to a worker hosting them. Call before `run`. + * The tools are registered once on the client, for every agent: the router offers one to + * a session only where a tool group of its agent's agent.yaml names it, and sends every + * call of it to a worker hosting it. `client.runDispatch()` does this alone, for a worker + * with no calls or messages to answer. Call before `run`. * * ```ts - * const agent = client.agent("my-agent"); - * agent.tools.register({ name: "lookup", description: "Look up an order", run }); - * await new Dispatch().host(agent).run(); + * client.tools.register({ name: "lookup", description: "Look up an order", run }); + * await new Dispatch({ client }).host(client).run(); * ``` */ - host(agent: AgentHandle, options: HostOptions = {}): this { - const offer = { - agentId: agent.name, - tools: agent.tools, - toolTimeoutMs: options.toolTimeoutMs ?? 0, - }; + host(client: Client, options: HostOptions = {}): this { + const offer = { tools: client.tools, toolTimeoutMs: options.toolTimeoutMs ?? 0 }; this.checkHosting([...this.hosted, offer]); this.hosted.push(offer); return this; @@ -408,7 +401,7 @@ export class Dispatch { // The router took the offer. Nothing changes here: the calls simply start. break; case "hosting_refused": - throw new HostingRefusedError(text(message, "agent_id"), text(message, "reason")); + throw new HostingRefusedError(text(message, "reason")); case "pong": this.pong?.(numberOf(message, "at")); break; @@ -505,7 +498,7 @@ export class Dispatch { for (const offer of this.hosted) { this.tell({ type: "host_tools", - agent_id: offer.agentId, + every_agent: true, tools: offer.tools.declared(), timeout_ms: offer.toolTimeoutMs, }); diff --git a/sdks/js/src/errors.ts b/sdks/js/src/errors.ts index f680017e5a..3fb868cc3e 100644 --- a/sdks/js/src/errors.ts +++ b/sdks/js/src/errors.ts @@ -66,15 +66,12 @@ export class SocketClosedError extends Error { * again would only be told the same thing. */ export class HostingRefusedError extends Error { - /** The agent the tools were offered for. */ - readonly agentId: string; /** Why the router said no. */ readonly reason: string; - constructor(agentId: string, reason: string) { - super(`the router refused to host tools for agent ${agentId}: ${reason}`); + constructor(reason: string) { + super(`the router refused to host tools: ${reason}`); this.name = "HostingRefusedError"; - this.agentId = agentId; this.reason = reason; } } diff --git a/sdks/js/src/generated/api.ts b/sdks/js/src/generated/api.ts index 5d5f03ee8f..c6d1b96e3d 100644 --- a/sdks/js/src/generated/api.ts +++ b/sdks/js/src/generated/api.ts @@ -2091,12 +2091,13 @@ export type paths = { /** * Watch the conversation and answer the model's tool calls * @description A WebSocket, which OpenAPI cannot describe past the upgrade. Frames are JSON objects carrying a `type` and the fields of that event. - * The server sends what the conversation did: `joined`, `heard`, `responding`, `response_delta`, `reasoning_delta` (turn_id and text: the next piece of the model's thinking on a written reply, never spoken), `responded` (pending_work remains true while tools or delegated work are outstanding), `said` (a line the agent says as written rather than asking the model, such as its greeting, with the purpose it has), `speaking` (an utterance started being heard: turn_id, synthesis_id, the text the voice had been sent for it by then, at, when the call's track took its first audible frame, and delay_ms, for an utterance reported before the track got to it, how long after at it is expected to be heard, from the speech queued ahead of it, and message_start and message_end, where that text is in the reply's Chat message, whose text is not the voice's: directions a voice acts, digits it is told the count of and sentences held back are in one and not the other; a streaming voice's grows as it is sent more), `spoke` (an utterance finished being synthesised: turn_id, synthesis_id, characters (Unicode code points, as every character count on this socket is), audio_duration_ms, time_to_first_byte_ms, and message_start and message_end for all of it; it follows the utterance's `speaking`), `turn`, `decision`, `delegated`, `task_settled` (files lists what the work's code handed back, each a name, mime_type, url and size, uploaded to a persistent conversation's channel and attached to the reply), `task_cancelled`, `tool_call`, `tool_held`, `tool_approval_ignored`, `tool_updated`, `tool_ran`, `transferred`, `pressed`, `looked_up`, `backchannel`, `interrupted` (turn_id; with synthesis_id, heard_ms, message_start and message_end, the utterance being heard when it was cut, how many milliseconds of it had been, and where all it had been sent is in the reply's message, when any of it had), `overlap_decided`, `conversation_compacted`, `models_changed`, `error` and `left`. + * The server sends what the conversation did: `joined`, `heard`, `responding`, `response_delta`, `reasoning_delta` (turn_id and text: the next piece of the model's thinking on a written reply, never spoken), `responded` (pending_work remains true while tools or delegated work are outstanding), `said` (a line the agent says as written rather than asking the model, such as its greeting, with the purpose it has), `speaking` (an utterance started being heard: turn_id, synthesis_id, the text the voice had been sent for it by then, at, when the call's track took its first audible frame, and delay_ms, for an utterance reported before the track got to it, how long after at it is expected to be heard, from the speech queued ahead of it, and message_start and message_end, where that text is in the reply's Chat message, whose text is not the voice's: directions a voice acts, digits it is told the count of and sentences held back are in one and not the other; a streaming voice's grows as it is sent more), `spoke` (an utterance finished being synthesised: turn_id, synthesis_id, characters (Unicode code points, as every character count on this socket is), audio_duration_ms, time_to_first_byte_ms, and message_start and message_end for all of it; it follows the utterance's `speaking`), `turn`, `decision`, `delegated`, `task_settled` (files lists what the work's code handed back, each a name, mime_type, url and size, uploaded to a persistent conversation's channel and attached to the reply), `task_cancelled`, `tool_call`, `tool_held`, `tool_approval_ignored`, `tool_started`, `tool_updated`, `tool_ran`, `tool_call.started`, `tool_call.progress`, `tool_call.finished`, `transferred`, `pressed`, `looked_up`, `backchannel`, `interrupted` (turn_id; with synthesis_id, heard_ms, message_start and message_end, the utterance being heard when it was cut, how many milliseconds of it had been, and where all it had been sent is in the reply's message, when any of it had), `overlap_decided`, `conversation_compacted`, `models_changed`, `error` and `left`. * An `error` frame for a turn the prompt injection screen refused carries `code` `prompt_injection`, and the turn's `responded` has empty text: what streamed before the verdict is not the answer. * message_start and message_end are offsets in Unicode code points into the text of the Chat message the reply is written into, the one whose turn_id names the turn, from message_start up to message_end. They are left out where the router does not know them. * `connector_unavailable` names an optional connector binding the session opened without: name (its alias), connector_id and reason, one of no_selection, shared_session, caller_unverified, connection_unavailable, provider_mismatch, needs_reauthorization, credential_rejected (the provider rejected the token or key a bearer or api_key connection holds, or the connection reads a connector revision marked broken; only saving its credentials again fixes it, so no login is offered), not_connected, open_failed, tool_unavailable and selection_dropped (a fork's or a reopened chat's selection for an alias its config no longer declares). Every watcher is sent each one when it attaches. * `connector_scope_required` says a connector tool call was refused because the caller's own connection lacks access the provider asked for (insufficient_scope or a claims challenge), and a step-up consent was begun for it: name (the binding's alias), connector_id, connection_id, scopes (what the provider asked for, empty for a claims challenge), authorization_id, launch_url, handoff_token and expires_at. A page of the session's return_url's origin, or the dashboard for a session created without one, opens launch_url in a popup and posts it handoff_token, as for a createAuthorization with that return_url: the launch page takes the handoff from that origin only, and the consent goes back to the session's return_url, or else to the dashboard. The old grant keeps working until the step-up succeeds, and the same call works afterwards in the same session. While that step-up is open, calls refused for the same access send no second event. * Persistent text sessions also emit `conversation_updated` with conversation_id and a complete message snapshot: id, request_id, question_id, role, text, state, response_started_at, state_started_at, finished_at, duration_ms, saved, persistence_error, refused (true on a reply the prompt injection screen refused) and attachments. Each tool_calling attachment has tool_call_id, name, title, status, phase, summary, immutable started_at, execution_started_at, finished_at and duration_ms. A connector_authorization attachment asks the end user to connect a connector binding the reply needed with their own account, with name (the binding's alias), connector_id, connection_id, authorization_id, title, launch_url, handoff_token and expires_at: a page of the session's return_url's origin, or the dashboard for a session created without one, opens launch_url in a popup and posts it handoff_token, as for a createAuthorization with that return_url, and the consent goes back to the session's return_url, or else to the dashboard. Once the user finishes that login the message is sent again with status connected and no handoff_token, and the agent carries on by itself. The call that needed the login waits for it on its own ai_tool_call step, in parts: status awaiting_authorization, target_user_id the caller, and an authorization of type oauth with provider (the connector id), provider_name, title, message, url (the launch page), handoff_token and expires_at. A page of the return_url's origin opens url and posts it handoff_token; a native app opens url#handoff_token= in its web authentication sheet. Connected, the step is running again with authorization.status connected and no handoff_token, and the call runs in the same reply. A tool_approval naming its tool_call_id with allowed false, or the person refusing at the provider, declines the login (declined); a consent that ended otherwise without connecting, such as with another account, is failed, and one not finished by expires_at is expired. Each way the step is cancelled, authorization.status says how, and the call runs nothing. Activity states are thinking, queued, tools, writing, completed, failed and cancelled. tool_started includes tool_call_id, tool, turn_id and started_at, and pre_speech when the tool's connector binding sets one in its policy; tool_ran also includes tool_call_id, and files when the tool was run_code, as on task_settled. + * `tool_call.started`, `tool_call.progress` and `tool_call.finished` (`ToolCallState`) are a call's state as a client shows it, whoever runs the call: tool_call_id, tool, turn_id, status (in_progress, waiting for a person's approval, success, failed or cancelled), progress (the last `tool_update`, the binding's pre_speech or the approval question), started_at, duration_ms, approved_at, rejected_at and error. Each carries the whole state: keep the last per tool_call_id. * A respond command carrying request_id emits command_accepted with a nested command receipt (request_id, user_message_id, assistant_message_id, state, duplicate). Personal persistent text sessions require this ID. A retry with the same text and images returns the existing IDs without invoking the model again; reuse with different content emits an error. Its images are data URIs, at most 8, uploaded to the conversation's channel and shown on the person's message. On a session a dispatch worker opened for a message handed out from a channel, that message is already there: nothing is written or uploaded for it, user_message_id is empty, and a reused id is answered as a duplicate even with other text or images. After restart an interrupted command is reported, not rerun. * An `interrupt` command carrying `request_id` stops that command and emits `command_stopped` with its terminal receipt. A stop arriving after its command finished replays that command's receipt and leaves the command running now alone; an unknown command is reported as an error. Without `request_id` the frame stops whichever reply is current, which is what a caller with no command to name means by it. * A `decision` frame is one judgement the conversation made, carrying the same fields as a CallEvent. Together they are why the call went the way it did, and they are also written down, so a finished call replays them from `/v1/agents/calls/{id}/events`. @@ -4347,7 +4348,8 @@ export type paths = { }; /** * How the calling customer's agents ran, across all of them - * @description The most agents that were running at once, and which tools they called: how often, how much each call put in front of the model and how long it took. + * @description The most agents that were running at once, and which tools they called: how often, in how many sessions and after how many turns, how much each call put in front of the model, what the tool's definition costs every turn that carries it and how long a call took. + * With agent, the sessions, turns and tools are that agent config's alone; the peak is still every agent's. * One figure for the whole window rather than a row per bucket, because neither a peak nor a percentile can be summed out of smaller ones. */ readonly get: operations["getAgentStats"]; @@ -4904,7 +4906,8 @@ export type components = { readonly tags?: { readonly [key: string]: string; }; - readonly tools?: components["schemas"]["AgentTools"]; + /** @description The functions an SDK provides that the agent may offer, in named groups. Empty when there are none, and then the agent offers no function. */ + readonly tools?: readonly components["schemas"]["ToolGroup"][]; readonly tts?: string; /** @description Absent when the agent is taught no UI. */ readonly ui_instructions?: string; @@ -4984,7 +4987,8 @@ export type components = { readonly tags?: { readonly [key: string]: string; }; - readonly tools?: components["schemas"]["AgentTools"]; + /** @description Replaces the tool groups whole; an empty list removes them all. */ + readonly tools?: readonly components["schemas"]["ToolGroup"][]; readonly tts?: string; /** @description What teaches the agent's model to answer with UI. An empty string removes them. */ readonly ui_instructions?: string; @@ -5075,8 +5079,8 @@ export type components = { readonly tags?: { readonly [key: string]: string; }; - /** @description How the agent is offered its connector tools. Left out on an update, the stored settings stay. */ - readonly tools?: components["schemas"]["AgentTools"]; + /** @description The functions an SDK provides that the agent may offer, in named groups, each with how it is loaded. A function an SDK registers, declares on a session or hosts in a worker is offered only when a group names it: a session declaring one that none names is refused (400 tool_not_configured), and a worker's is left out. The router's own tools, connector tools and other agents are not named here. Omitted or null on an update, the groups stored stay; an empty list removes them all. The mapping this field once was, with progressive and output_retention, is refused: loading summary and output_retention on a group or a connector binding replaced them. */ + readonly tools?: readonly components["schemas"]["ToolGroup"][]; readonly tts?: string; /** @description What teaches the agent's model to answer with UI, such as the section the SDK compiles from agent.yaml's ui key. Every session is told them after the instructions, unless it was opened with ui false. Empty teaches none. Left out on an update, the stored ones stay. */ readonly ui_instructions?: string; @@ -5093,8 +5097,15 @@ export type components = { readonly connector_id: string; /** @description MCP events the binding's fixed connection is subscribed to, each opening a text conversation from the config when it arrives. Subscribed when the connection is next validated. Only a fixed binding may declare events: a session binding's connection is picked when a session opens, and an event arrives with no session open. */ readonly events?: readonly components["schemas"]["ConnectorBindingEvent"][]; + /** @description How the binding's tools are offered to the model. Omitted is full. */ + readonly loading?: components["schemas"]["ToolLoading"]; /** @description The alias, unique within the config: a lowercase letter, then up to 62 lowercase letters, digits, - or _, never __ and not ending in _. The model is offered each tool as __, split back at the first __, so a __ inside the alias or a _ at its end would split it in the wrong place. */ readonly name: string; + /** + * @description How long the agent's sessions keep what the binding's tools were given and gave back on the turns they record, as a tool group's output_retention. Omitted or empty, it is kept as long as the session. + * @example 30d + */ + readonly output_retention?: string; /** @description How the binding's tool calls behave around speech and interruptions. Omitted, a call is cancelled at the provider when the turn is interrupted, and the agent picks its own words while it runs. */ readonly policy?: components["schemas"]["ConnectorBindingPolicy"]; /** @description Names or path.Match patterns of the binding's tools whose every call waits for a person to allow it before the router sends it, such as create_issue or delete_*. The step of the call asks the session's user in its conversation, who answers with the session socket's tool_approval. Declined, or not answered in 10 minutes, the call is not sent and the model is told so. A session with nobody to ask, such as an incognito one or one opened by an MCP event, refuses the call at once. Each entry must match a tool the binding grants, or one of its tool_patterns. Refused with sandbox, whose code calls the API with nobody asked. At most 128, the cap tools has. */ @@ -5243,6 +5254,11 @@ export type components = { readonly max_tasks_per_user?: number; }; readonly AgentStats: { + /** + * Format: double + * @description Turns a session took in the window, on average. A turn is a reply to the person, not a rewound one or a delegated task's. Null when no session took one. + */ + readonly avg_turns: number | null; /** * Format: date-time * @description When that peak was first reached. Null when no session was open in the window. @@ -5253,21 +5269,16 @@ export type components = { * @description The most sessions open at the same time, each one an agent running. A session still open counts up to now. */ readonly peak_concurrent_sessions: number; + /** + * Format: int64 + * @description Sessions that took a turn in the window: what a tool's session_share is out of. + */ + readonly sessions: number; /** @description Every tool call in the window, as one. Its tool is empty. */ readonly tool_calls: components["schemas"]["ToolStats"]; /** @description One row per tool, most called first. */ readonly tools: readonly components["schemas"]["ToolStats"][]; }; - /** @description How an agent is offered its tools, and how long what they were given and gave back is kept. */ - readonly AgentTools: { - /** - * @description How long this agent's sessions keep what any of its tools were given and gave back on the turns they record: a tool's arguments, its result, a search's query and the wording of a tool's error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed. Left out on a create, or empty, it is kept as long as the session; left out on an update, the stored setting stays, and an empty string removes it. The app's and the organization's policies (tool_output_retention) can shorten it and never lengthen it, and the shortest applies. It applies to tool calls recorded after it is set. - * @example 30d - */ - readonly output_retention?: string; - /** @description Offer each tool by the first line of its description, with its arguments' descriptions left out, and have the first call to a tool return its full description and input schema instead of running it. It saves context on an agent with many tools, at the cost of one more model turn for each tool a conversation uses. Off by default. Left out on an update, the stored setting stays. */ - readonly progressive?: boolean; - }; readonly AgentUser: { /** @description true when the user reads all of what the number ever said. Left out otherwise. */ readonly include_phone_history?: boolean; @@ -7411,7 +7422,7 @@ export type components = { readonly $in?: readonly string[]; }; readonly ErrorDetail: { - /** @description What went wrong, for a program to branch on. Every type has a code of its own name (invalid_request, unauthenticated, forbidden, payment_required, not_found, method_not_allowed, not_acceptable, conflict, gone, payload_too_large, unsupported_media_type, rate_limited, internal_error, unavailable, timeout) that a failure has when nothing names it better. The others are validation_failed, missing_customer, missing_organization, server_side_only, not_configured (this deployment does not offer the feature), modality_not_routed, unsynced_changes (a sync asked to check would write over somebody's edits), name_taken (a 409: another agent config, router config or voice, or another skill of the same agent config, already has the name, so another name will do), channel_connection_taken (a 409: another live agent config binds the channel connection as fixed, and one agent answers a channel connection's messages, so the message names that config), call_unanswered (a 409: the call placed for a phone session ended with nobody answering, and the message names how: busy, no_answer, failed or canceled. Opening the session is refused, or, when the status came while the agent was joining the call, the session is ended at once and kept as ended), no_oauth_client (a 400: a consent found no OAuth client for the connector, and the message names each way the connector lets the app give it one), invalid_config (a 400: an agent config a run names does not start a session), insufficient_credits (a 402: the organization's credit is spent, so work that costs money is refused until credit is added; a model that costs nothing is still served), system_limit_reached (a 402: the organization's spend hard limit for the day or the month is reached, whatever its credit, and the message names the interval, the amount and when it resets), spend_limit_reached (a 402: one of the organization's own spend limits is spent, on the organization, an app, a key, a usage category or a team, and the message says which and when it resets, so retrying before then is refused again), user_limit_reached (a 429: a spend limit on the one end user the request is for is spent, which stops that user and no one else, and Retry-After says when its window ends), spend_check_unavailable (a 503: the router could not share what the organization is spending with the other routers for a while, so work that costs money is refused until it can; a model that costs nothing is still served, and Retry-After says to try again in a few seconds), budget_moved (a 400: a policy sent a budget, which spend limits replaced), limit_above_parent (a 400: a spend limit is above a limit of its parent in the same interval, and the message names the parent's limit), limit_exists (a 409: the subject already has a limit at that level and interval, and the message names its id), no_limit (a 404: the subject has no spend limit to reset or grant past), use_team (a 400: a category starting team: was sent, where the team field names a team), invalid_expiry (a 400: a grant's expires_at is not in the future, or is more than 366 days away), too_many_grants (a 409: the organization has 20 live grants, as many as it may), tier_limit_reached (the organization's tier limits refuse the work, and limit says which: a 403 for a call or a transfer, to a country the tier does not reach, past its minutes or recipients of the day, or past its calls at once; a 400 for an agent config whose sandbox is bigger or more open than the organization's sandbox limits allow. The message names the tier and the limit), account_suspended (a 403: the organization is suspended, and every request is refused, one for a model that costs nothing included, until it is restored; the message says to contact support), concurrency_limited (a 429: the app has as many runs in flight as it may, and Retry-After says when to try again), model_failed (a 503: the model failed or gave no answer), prompt_injection (a 400: the prompt injection screen refused the input of a run or a tool result it read, which retrying an unchanged input does not fix), image_unavailable (a 400: an image a run was handed could not be fetched or is not a JPEG, PNG, WebP or GIF, which retrying the same image does not fix), document_too_large (a 413: a file sent with a response has more pages or text than one may, alone or with the files the conversation already holds, and the message names the limit), document_unreadable (a 400: a file sent with a response could not be read, such as one with a password, one that is damaged or one with no text in it, which retrying the same file does not fix), invalid_usage_categories (a 400: the usage categories the request names, in the token's usage_categories and team claims or in X-Stream-Usage-Categories, are more than 10, one is not 1 to 64 characters of A-Z, a-z, 0-9, _ . : -, more than one names a team, or the team is over 59 characters), memory_unavailable (a 503: the memory store could not delete what was asked, so nothing that waits on it was deleted either, and the same request later can succeed), billing_not_enabled (a 404: this deployment bills nobody, or the caller's app is in no organization it bills, so there is no account to read), provider_takes_no_key (a 400: a key was put for a provider whose work is never routed on a customer's own key here, and the message names the ones that are), campaign_locked (a 409: the campaign's state does not allow the change, such as a launched campaign's sequence, or resuming one that is running), campaign_not_ready (a 400: a launch found the campaign missing what it needs to ring anybody, and the message says what), consent_required (a 400: a launch did not say the people on the campaign agreed to be called), branch_limit_reached (a 409: the conversation has as many edit and regenerate points as Stream Chat keeps for it, 100, so a regenerate or an edit is refused and the conversation is continued in a new chat; asking and reading are unaffected), branch_too_long (a 409: the branch an activation would bring back is more than 100 messages to send again, so it is continued in a new chat), branch_unavailable (a 410: the copies of a hidden branch were deleted, as a memory truncate deletes them, so it can be neither read nor brought back), switch_unfinished (a 409: a regenerate, an edit or an activation of the session failed after Stream Chat may have rewound, and finishing it still fails because Stream Chat does not answer, so the session's questions, regenerates, edits and activations are refused meanwhile; try again: each one tries to finish it first, so it clears once Stream Chat answers), and _not_found for agent_config, call, campaign, channel_account, command, connection, grant, knowledge_document, knowledge_url, limit, provider_key, router_config, scheduled_task, session, simulation, simulation_run, skill and voice. A scheduled task also has invalid_schedule (a schedule that does not read: at and in both or neither, a date, time or duration that does not parse, or until without repeat), schedule_in_past, schedule_too_far (a one-shot, or an in, more than a year ahead), invalid_timezone, timezone_unknown (no zone was given and neither the user nor the config has one), invalid_repeat, interval_too_short (a run moved to less than 15 minutes from now and earlier than the run it replaces) and too_many_tasks (a 409: the user is at their active-task limit). More may be added, so a client should expect one it does not know. */ + /** @description What went wrong, for a program to branch on. Every type has a code of its own name (invalid_request, unauthenticated, forbidden, payment_required, not_found, method_not_allowed, not_acceptable, conflict, gone, payload_too_large, unsupported_media_type, rate_limited, internal_error, unavailable, timeout) that a failure has when nothing names it better. The others are validation_failed, missing_customer, missing_organization, server_side_only, not_configured (this deployment does not offer the feature), modality_not_routed, unsynced_changes (a sync asked to check would write over somebody's edits), name_taken (a 409: another agent config, router config or voice, or another skill of the same agent config, already has the name, so another name will do), channel_connection_taken (a 409: another live agent config binds the channel connection as fixed, and one agent answers a channel connection's messages, so the message names that config), call_unanswered (a 409: the call placed for a phone session ended with nobody answering, and the message names how: busy, no_answer, failed or canceled. Opening the session is refused, or, when the status came while the agent was joining the call, the session is ended at once and kept as ended), no_oauth_client (a 400: a consent found no OAuth client for the connector, and the message names each way the connector lets the app give it one), invalid_config (a 400: an agent config a run names does not start a session), tool_not_configured (a 400: a session under an agent config was declared or given a function that none of the config's tool groups names, and the message names it), insufficient_credits (a 402: the organization's credit is spent, so work that costs money is refused until credit is added; a model that costs nothing is still served), system_limit_reached (a 402: the organization's spend hard limit for the day or the month is reached, whatever its credit, and the message names the interval, the amount and when it resets), spend_limit_reached (a 402: one of the organization's own spend limits is spent, on the organization, an app, a key, a usage category or a team, and the message says which and when it resets, so retrying before then is refused again), user_limit_reached (a 429: a spend limit on the one end user the request is for is spent, which stops that user and no one else, and Retry-After says when its window ends), spend_check_unavailable (a 503: the router could not share what the organization is spending with the other routers for a while, so work that costs money is refused until it can; a model that costs nothing is still served, and Retry-After says to try again in a few seconds), budget_moved (a 400: a policy sent a budget, which spend limits replaced), limit_above_parent (a 400: a spend limit is above a limit of its parent in the same interval, and the message names the parent's limit), limit_exists (a 409: the subject already has a limit at that level and interval, and the message names its id), no_limit (a 404: the subject has no spend limit to reset or grant past), use_team (a 400: a category starting team: was sent, where the team field names a team), invalid_expiry (a 400: a grant's expires_at is not in the future, or is more than 366 days away), too_many_grants (a 409: the organization has 20 live grants, as many as it may), tier_limit_reached (the organization's tier limits refuse the work, and limit says which: a 403 for a call or a transfer, to a country the tier does not reach, past its minutes or recipients of the day, or past its calls at once; a 400 for an agent config whose sandbox is bigger or more open than the organization's sandbox limits allow. The message names the tier and the limit), account_suspended (a 403: the organization is suspended, and every request is refused, one for a model that costs nothing included, until it is restored; the message says to contact support), concurrency_limited (a 429: the app has as many runs in flight as it may, and Retry-After says when to try again), model_failed (a 503: the model failed or gave no answer), prompt_injection (a 400: the prompt injection screen refused the input of a run or a tool result it read, which retrying an unchanged input does not fix), image_unavailable (a 400: an image a run was handed could not be fetched or is not a JPEG, PNG, WebP or GIF, which retrying the same image does not fix), document_too_large (a 413: a file sent with a response has more pages or text than one may, alone or with the files the conversation already holds, and the message names the limit), document_unreadable (a 400: a file sent with a response could not be read, such as one with a password, one that is damaged or one with no text in it, which retrying the same file does not fix), invalid_usage_categories (a 400: the usage categories the request names, in the token's usage_categories and team claims or in X-Stream-Usage-Categories, are more than 10, one is not 1 to 64 characters of A-Z, a-z, 0-9, _ . : -, more than one names a team, or the team is over 59 characters), memory_unavailable (a 503: the memory store could not delete what was asked, so nothing that waits on it was deleted either, and the same request later can succeed), billing_not_enabled (a 404: this deployment bills nobody, or the caller's app is in no organization it bills, so there is no account to read), provider_takes_no_key (a 400: a key was put for a provider whose work is never routed on a customer's own key here, and the message names the ones that are), campaign_locked (a 409: the campaign's state does not allow the change, such as a launched campaign's sequence, or resuming one that is running), campaign_not_ready (a 400: a launch found the campaign missing what it needs to ring anybody, and the message says what), consent_required (a 400: a launch did not say the people on the campaign agreed to be called), branch_limit_reached (a 409: the conversation has as many edit and regenerate points as Stream Chat keeps for it, 100, so a regenerate or an edit is refused and the conversation is continued in a new chat; asking and reading are unaffected), branch_too_long (a 409: the branch an activation would bring back is more than 100 messages to send again, so it is continued in a new chat), branch_unavailable (a 410: the copies of a hidden branch were deleted, as a memory truncate deletes them, so it can be neither read nor brought back), switch_unfinished (a 409: a regenerate, an edit or an activation of the session failed after Stream Chat may have rewound, and finishing it still fails because Stream Chat does not answer, so the session's questions, regenerates, edits and activations are refused meanwhile; try again: each one tries to finish it first, so it clears once Stream Chat answers), and _not_found for agent_config, call, campaign, channel_account, command, connection, grant, knowledge_document, knowledge_url, limit, provider_key, router_config, scheduled_task, session, simulation, simulation_run, skill and voice. A scheduled task also has invalid_schedule (a schedule that does not read: at and in both or neither, a date, time or duration that does not parse, or until without repeat), schedule_in_past, schedule_too_far (a one-shot, or an in, more than a year ahead), invalid_timezone, timezone_unknown (no zone was given and neither the user nor the config has one), invalid_repeat, interval_too_short (a run moved to less than 15 minutes from now and earlier than the run it replaces) and too_many_tasks (a 409: the user is at their active-task limit). More may be added, so a client should expect one it does not know. */ readonly code: string; /** * Format: uri @@ -8454,6 +8465,8 @@ export type components = { readonly OfferedTool: { /** @description What the model is told the tool does. */ readonly description: string; + /** @description How the tool is offered, as its tool group or connector binding says: full for the router's own. A deferred tool is listed once the model has loaded it. Absent for a session that has ended. */ + readonly loading?: components["schemas"]["ToolLoading"]; /** @description How the model asks for it. */ readonly name: string; /** @description The JSON Schema of its arguments. */ @@ -8678,7 +8691,7 @@ export type components = { readonly [key: string]: string; }; /** - * @description How long a session keeps what its tools were given and gave back on the turns it records: a tool's arguments, its result, a search's query and the wording of a tool's error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed, so a session's items read in the same order and tool usage still counts the call. Left out, or empty, it is kept as long as the session. The shortest of the organization's, the app's and the agent config's tools.output_retention applies, so an agent can shorten it and never lengthen it. It applies to tool calls recorded after it is set. Deleting a session deletes all of it at once. + * @description How long a session keeps what its tools were given and gave back on the turns it records: a tool's arguments, its result, a search's query and the wording of a tool's error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed, so a session's items read in the same order and tool usage still counts the call. Left out, or empty, it is kept as long as the session. The shortest of the organization's, the app's and the output_retention of the agent config's tool group or connector binding the tool is in applies, so an agent can shorten it and never lengthen it. It applies to tool calls recorded after it is set. Deleting a session deletes all of it at once. * @example 30d */ readonly tool_output_retention?: string; @@ -10680,8 +10693,8 @@ export type components = { readonly tags?: { readonly [key: string]: string; }; - /** @description How connector tools are offered. A setting left out keeps what is stored. */ - readonly tools?: components["schemas"]["AgentTools"]; + /** @description agent.yaml's tools: the functions an SDK provides that the agent may offer, in named groups, as AgentConfigRequest's tools names them, with the same rules. Sent, they replace the stored groups whole, an empty list removing them. Left out, the stored ones stay. */ + readonly tools?: readonly components["schemas"]["ToolGroup"][]; readonly tts?: string; /** @description What teaches the agent's model to answer with UI, which the SDK compiles from agent.yaml's ui key. Kept apart from instructions, so a session opened with ui false is not told them. Left out, the agent is taught none. */ readonly ui_instructions?: string; @@ -10917,6 +10930,65 @@ export type components = { /** @enum {string} */ readonly type: "tool_approval"; }; + /** + * @description tool_call.started is the call's first frame, tool_call.progress one while it runs or waits, and tool_call.finished its last. + * @enum {string} + */ + readonly ToolCallFrameType: "tool_call.started" | "tool_call.progress" | "tool_call.finished"; + /** @description One change to a tool call's state. Every watcher is sent tool_call.started once, tool_call.progress for each change while it runs or waits, and tool_call.finished once, carrying the call's whole state each time, so a client keeps the last one it got per tool_call_id. */ + readonly ToolCallState: { + /** + * Format: date-time + * @description When a person allowed the call. + */ + readonly approved_at?: string; + /** + * Format: int64 + * @description From started_at to this change; on tool_call.finished, how long the call took. + */ + readonly duration_ms: number; + /** @description Why a failed call failed, or what a declined one shows, such as "Not allowed in time". */ + readonly error?: string; + /** @description The last thing said about the call while it runs, such as "Searching for flights": a tool_update, the connector binding's pre_speech, or the question it waits on. */ + readonly progress?: string; + /** + * Format: date-time + * @description When a person declined the call, or its wait for them passed. + */ + readonly rejected_at?: string; + /** Format: date-time */ + readonly started_at: string; + readonly status: components["schemas"]["ToolCallStatus"]; + readonly tool: string; + /** @description The call, as tool_call, tool_started and tool_ran name it. */ + readonly tool_call_id: string; + readonly turn_id: string; + readonly type: components["schemas"]["ToolCallFrameType"]; + }; + /** + * @description in_progress while it runs, waiting while it is held for a person's approval, and success, failed or cancelled once it finished. A call declined, or whose approval nobody gave in time, is cancelled with rejected_at. + * @enum {string} + */ + readonly ToolCallStatus: "in_progress" | "waiting" | "success" | "failed" | "cancelled"; + /** @description A named set of functions an SDK provides that the agent may offer, how they are loaded and how long what they were given and gave back is kept. A function an SDK registers, declares on a session or hosts in a worker is offered only when a group names it. */ + readonly ToolGroup: { + readonly loading?: components["schemas"]["ToolLoading"]; + /** @description What the group is called, unique within the config. */ + readonly name: string; + /** + * @description How long the agent's sessions keep what the group's tools were given and gave back on the turns they record: a tool's arguments, its result and the wording of its error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed. Left out or empty, it is kept as long as the session. The app's and the organization's policies (tool_output_retention) can shorten it and never lengthen it. It applies to tool calls recorded after it is set. + * @example 30d + */ + readonly output_retention?: string; + /** @description The functions' names, as the SDK registers them. A name sits in one group at most. */ + readonly tools: readonly string[]; + }; + /** + * @description How a tool is offered to the model. full sends its whole description and input schema from the start. summary sends the first line of its description and its arguments without their descriptions, and the first call returns the full description and schema instead of running it: one more model turn for each tool a conversation uses, for less context on every turn. deferred offers nothing until the model needs it: an agent with a deferred tool is offered search_tool, which finds deferred tools by what they do, and load_tool, which returns their full descriptions and offers them from the model's next step on. + * @default full + * @enum {string} + */ + readonly ToolLoading: "full" | "summary" | "deferred"; readonly ToolResultCommand: { readonly error?: string; readonly output?: components["schemas"]["MessageContent"]; @@ -10937,15 +11009,42 @@ export type components = { * @description What a call put in front of the model, its arguments and its result, estimated at four characters a token. */ readonly avg_tokens: number; + /** + * Format: double + * @description Turns a session took before its first call, on average: 0 is a call on the first turn. Null on the row over every tool. + */ + readonly avg_turns_before_use: number | null; /** Format: int64 */ readonly calls: number; + /** + * Format: int64 + * @description What the tool's whole definition, its name, description and schema, costs every turn that carries it, estimated at four characters a token. Null when no call recorded it. + */ + readonly definition_tokens: number | null; /** * Format: int64 * @description Calls that ended in an error. */ readonly failed: number; + /** @description How the latest call had the tool offered: full, summary or deferred. Empty when no call recorded it, as calls made before it was recorded and built-in tools. */ + readonly loading: string; /** Format: double */ readonly p99_ms: number | null; + /** + * Format: double + * @description Sessions that called it, out of the window's sessions, from 0 to 1. Null when no session took a turn. + */ + readonly session_share: number | null; + /** + * Format: int64 + * @description Sessions that called it at least once. + */ + readonly sessions: number; + /** + * Format: int64 + * @description What its definition costs offered as a summary. Null unless its loading is summary. + */ + readonly summary_tokens: number | null; readonly tool: string; }; /** @description Something the recording named, for the providers that pick them out. */ @@ -21978,6 +22077,8 @@ export interface operations { readonly getAgentStats: { readonly parameters: { readonly query: { + /** @description The id of an agent config: only its sessions are counted. Omitted is every agent. */ + readonly agent?: string; /** @description Start of the window, inclusive. */ readonly from: string; /** @description End of the window, exclusive. */ @@ -22006,6 +22107,7 @@ export interface operations { readonly 401: components["responses"]["Unauthorized"]; readonly 402: components["responses"]["PaymentRequired"]; readonly 403: components["responses"]["Forbidden"]; + readonly 404: components["responses"]["NotFound"]; readonly 500: components["responses"]["InternalError"]; readonly 503: components["responses"]["Unavailable"]; }; diff --git a/sdks/js/src/handle.ts b/sdks/js/src/handle.ts index 7b535cb60b..651c9c8530 100644 --- a/sdks/js/src/handle.ts +++ b/sdks/js/src/handle.ts @@ -1,7 +1,6 @@ import type { Client, Schemas } from "./client.js"; import { ConfigurationError } from "./errors.js"; import { Sessions } from "./sessions.js"; -import { Tools } from "./tools.js"; /** * An agent, addressed by the name it is configured under. @@ -24,11 +23,6 @@ export class AgentHandle { readonly name: string; /** This agent's conversations: opening one, and reading the old ones back. */ readonly sessions: Sessions; - /** - * Functions run for every session of this agent, whoever opened it, once a dispatch worker - * hosts them with `dispatch.host(agent)`. - */ - readonly tools = new Tools(); private readonly client: Client; diff --git a/sdks/js/src/index.ts b/sdks/js/src/index.ts index 535b809570..be04f501df 100644 --- a/sdks/js/src/index.ts +++ b/sdks/js/src/index.ts @@ -155,6 +155,7 @@ export { } from "./ui-catalog.js"; export { ConversationState, type AgentState, type ConversationSnapshot } from "./state.js"; +export { ToolCalls, type ToolCallState, type ToolCallStatus } from "./tool-calls.js"; export { Conversation, mergeOpenUIStatements, diff --git a/sdks/js/src/react.ts b/sdks/js/src/react.ts new file mode 100644 index 0000000000..502d833f7b --- /dev/null +++ b/sdks/js/src/react.ts @@ -0,0 +1,35 @@ +/** + * React hooks over a session's stores, opted into by importing this entry: + * + * ```ts + * import { useToolCall, useToolCalls } from "@stream-io/stream-agents/react"; + * ``` + * + * Its own entry point so `react` is resolved only for an app that asked for it. + */ + +import { useCallback, useEffect, useRef, useSyncExternalStore } from "react"; + +import type { Session, SessionEvent } from "./session.js"; +import type { ToolCallState } from "./tool-calls.js"; + +/** Every tool call of the session by `tool_call_id`, in the order they started, re-rendering as one changes. */ +export function useToolCalls(session: Session): ReadonlyMap { + return useSyncExternalStore(session.toolCalls.subscribe, session.toolCalls.get, session.toolCalls.get); +} + +/** One tool call as it was last reported, or undefined before the session said anything of it. */ +export function useToolCall(session: Session, id: string): ToolCallState | undefined { + const read = useCallback(() => session.toolCalls.get().get(id), [session, id]); + return useSyncExternalStore(session.toolCalls.subscribe, read, read); +} + +/** + * Calls `handler` with every event of `kind` while the component is mounted. The latest + * `handler` is the one called, so it need not be memoised. + */ +export function useSessionEvent(session: Session, kind: string, handler: (event: SessionEvent) => void): void { + const latest = useRef(handler); + latest.current = handler; + useEffect(() => session.on(kind, (event) => latest.current(event)), [session, kind]); +} diff --git a/sdks/js/src/session.ts b/sdks/js/src/session.ts index 9cd30ec3be..6a3a533712 100644 --- a/sdks/js/src/session.ts +++ b/sdks/js/src/session.ts @@ -7,6 +7,7 @@ import { Responses } from "./responses.js"; import { Socket, type Frame, flag, nested, text } from "./socket.js"; import { Speech } from "./speech.js"; import { ConversationState } from "./state.js"; +import { ToolCalls, type ToolCallState } from "./tool-calls.js"; import type { ChatClient, VideoClient } from "./stream.js"; import { Holds, socketLine, toolCallContextOf } from "./tool_line.js"; import { type ToolApproval, Tools } from "./tools.js"; @@ -76,9 +77,11 @@ export interface Participant { * * `kind` is the backend's own name for it: joined, hearing, heard, decision, responding, * response_delta, responded, blocked, speaking, spoke, turn, delegated, task_settled, - * task_cancelled, tool_started, tool_ran, transferred, pressed, looked_up, backchannel, - * interrupted, overlap_decided, conversation_compacted, conversation_updated, error and left, - * and tool_call for a call waiting in `session.approvals`. The fields + * task_cancelled, tool_started, tool_updated, tool_ran, tool_call.started, tool_call.progress + * and tool_call.finished (each call's state, also kept in `session.toolCalls`), transferred, + * pressed, looked_up, backchannel, interrupted, overlap_decided, conversation_compacted, + * conversation_updated, error and left, and tool_call for a call waiting in + * `session.approvals`. The fields * below are filled from whichever of those carry them, and `frame` is the whole thing for * anything they do not cover. * @@ -197,6 +200,12 @@ export class Session { * socket beside `events()`, like `state`. */ readonly conversation: Conversation; + /** + * Every tool call of the session by `tool_call_id`, as it was last reported: running, + * waiting for a person's approval, or how it ended, whichever process runs it. Fed from the + * socket beside `events()`, like `state`. + */ + readonly toolCalls = new ToolCalls(); /** * @internal What this device's camera showed lately, set by a `VoiceSession` while its * camera is on. The router's `get_video_frames` is answered from it. @@ -233,6 +242,8 @@ export class Session { /** What arrived while nobody was reading, for the first reader to start with. */ private readonly backlog: SessionEvent[] = []; private readonly readers = new Set(); + /** What `on` registered, by the kind of event each wants. */ + private readonly handlers = new Map void>>(); private readonly running = new Map(); private readonly holds = new Holds(); /** Aborts when the socket ends, which stops even a call that asked to be held. */ @@ -493,6 +504,30 @@ export class Session { } } + /** + * Calls `handler` with every event of `kind` that `events()` yields, such as + * `tool_call.finished` or `responded`, from now until the returned function is called. It + * needs no loop over `events()` and takes nothing from one. + */ + on(kind: string, handler: (event: SessionEvent) => void): () => void { + let handlers = this.handlers.get(kind); + if (handlers === undefined) { + handlers = new Set(); + this.handlers.set(kind, handlers); + } + const added = handlers; + const handle = (event: SessionEvent): void => handler(event); + added.add(handle); + return () => { + added.delete(handle); + }; + } + + /** The tool call `id` as it was last reported, or undefined before the session said anything of it. */ + toolCall(id: string): ToolCallState | undefined { + return this.toolCalls.get().get(id); + } + /** Speaks text without going through the model, for when you know what should be said. */ say(said: string, options: { interrupt?: boolean } = {}): void { if (options.interrupt) { @@ -808,6 +843,7 @@ export class Session { this.speech.observe(message); this.state.observe(message); this.conversation.observe(message); + this.toolCalls.observe(message); if (message.type === "tool_call" && text(message, "name") === CameraFrames.toolName) { // Answered by the session rather than offered as a tool, and only by a device whose @@ -1100,6 +1136,16 @@ export class Session { } private deliver(event: SessionEvent): void { + for (const handler of [...(this.handlers.get(event.kind) ?? [])]) { + try { + handler(event); + } catch (cause) { + // A handler runs on the socket's read, which one mistake must not stop. + queueMicrotask(() => { + throw cause; + }); + } + } if (this.readers.size === 0) { this.backlog.push(event); if (this.backlog.length > BUFFERED_EVENTS) { diff --git a/sdks/js/src/tool-calls.ts b/sdks/js/src/tool-calls.ts new file mode 100644 index 0000000000..5ce72dd951 --- /dev/null +++ b/sdks/js/src/tool-calls.ts @@ -0,0 +1,83 @@ +import type { Schemas } from "./client.js"; +import { number, text, type Frame } from "./socket.js"; + +/** One tool call as the session last reported it, whichever process runs it. */ +export type ToolCallState = Schemas["ToolCallState"]; + +/** `in_progress`, `waiting` for a person's approval, `success`, `failed` or `cancelled`. */ +export type ToolCallStatus = Schemas["ToolCallStatus"]; + +const frameTypes: ReadonlySet = new Set(["tool_call.started", "tool_call.progress", "tool_call.finished"]); + +/** + * The session's tool calls by `tool_call_id`, in the order they started, each as the session + * last reported it: `session.toolCalls`. Finished calls stay, with how they ended. + * + * Fed beside `events()`, like `session.state`. `get` and `subscribe` are bound, and the map is + * the same object until a call changes: + * + * ```ts + * const calls = useSyncExternalStore(session.toolCalls.subscribe, session.toolCalls.get); + * ``` + */ +export class ToolCalls { + private snapshot: ReadonlyMap = new Map(); + private readonly listeners = new Set<() => void>(); + + /** Every call so far. */ + readonly get = (): ReadonlyMap => this.snapshot; + + /** Calls `listener` whenever a call changes. It returns what stops it. */ + readonly subscribe = (listener: () => void): (() => void) => { + this.listeners.add(listener); + return () => { + this.listeners.delete(listener); + }; + }; + + /** @internal Folds one frame off the session's socket in. Frames about no call's state leave it alone. */ + observe(frame: Frame): void { + const state = toolCallStateOf(frame); + if (state === undefined) { + return; + } + const next = new Map(this.snapshot); + next.set(state.tool_call_id, state); + this.snapshot = next; + for (const listener of [...this.listeners]) { + try { + listener(); + } catch (cause) { + // The session's socket is read on this call, and one listener's mistake must not + // stop it. Reported the way a DOM event listener's is, without the others missing out. + queueMicrotask(() => { + throw cause; + }); + } + } + } +} + +/** The state a `tool_call.*` frame carries, or undefined for any other frame. */ +export function toolCallStateOf(frame: Frame): ToolCallState | undefined { + const type = text(frame, "type"); + const id = text(frame, "tool_call_id"); + if (!frameTypes.has(type) || id === "") { + return undefined; + } + const optional = (key: "progress" | "approved_at" | "rejected_at" | "error") => + text(frame, key) === "" ? {} : { [key]: text(frame, key) }; + return Object.freeze({ + type: type as ToolCallState["type"], + tool_call_id: id, + tool: text(frame, "tool"), + turn_id: text(frame, "turn_id"), + status: text(frame, "status") as ToolCallStatus, + started_at: text(frame, "started_at"), + duration_ms: number(frame, "duration_ms"), + ...optional("progress"), + ...optional("approved_at"), + ...optional("rejected_at"), + ...optional("error"), + }); +} diff --git a/sdks/js/src/tool_line.ts b/sdks/js/src/tool_line.ts index a8b8e2a3ed..1cf67d81e1 100644 --- a/sdks/js/src/tool_line.ts +++ b/sdks/js/src/tool_line.ts @@ -56,7 +56,6 @@ export class Holds { /** A call's way back to its session, over the socket that delivered it. */ export function socketLine(send: (frame: Record) => void, holds: Holds): ToolLine { return { - update: (id, said, verbatim) => send({ type: "tool_update", tool_call_id: id, text: said, verbatim }), hold: (id, signal) => holds.ask(id, () => send({ type: "tool_hold", tool_call_id: id }), signal), }; } diff --git a/sdks/js/src/tools.ts b/sdks/js/src/tools.ts index ee3a507e6a..c3a5a955ac 100644 --- a/sdks/js/src/tools.ts +++ b/sdks/js/src/tools.ts @@ -128,8 +128,6 @@ export interface ToolCallFields { * provides one; a test of a function's progress can too. */ export interface ToolLine { - /** Says how the call is going, or with `verbatim`, words to say as written. */ - update(id: string, text: string, verbatim: boolean): void; /** Asks for the call to keep going when the caller talks over the agent; resolves with whether it was still running to be held. */ hold(id: string, signal?: AbortSignal): Promise; } @@ -172,27 +170,6 @@ export class ToolCallContext implements ToolCallFields { this.lasting = lasting ?? new AbortController().signal; } - /** - * Says how the call is going. A voice agent tells the caller in its own words once nobody - * is talking, and a conversation shows it on the call's step until the call ends. A newer - * update replaces one not yet said, and the result replaces both. - */ - update(text: string): void { - if (this.line && this.id) { - this.line.update(this.id, text, false); - } - } - - /** - * Has the agent say `text` as written while the call runs, or post it in a text - * conversation. A speech-to-speech model is asked for these words and may paraphrase them. - */ - say(text: string): void { - if (this.line && this.id) { - this.line.update(this.id, text, true); - } - } - /** * Keeps the call going from now on when the caller talks over the agent, which is what a * write that must not be left halfway wants: from then on only the call's timeout and its diff --git a/sdks/js/tests/dispatch.test.ts b/sdks/js/tests/dispatch.test.ts index 343e10ff2b..9572921194 100644 --- a/sdks/js/tests/dispatch.test.ts +++ b/sdks/js/tests/dispatch.test.ts @@ -62,9 +62,8 @@ describe("Dispatch", () => { both.stop(); await running; - const agent = api.agent("stream-support"); - agent.tools.register({ name: "lookup", description: "Look up", run: () => "" }); - const hosting = new Dispatch({ client: api }).host(agent); + api.tools.register({ name: "lookup", description: "Look up", run: () => "" }); + const hosting = new Dispatch({ client: api }).host(api); running = hosting.run(); connection = await router.socket(); assert.equal( @@ -131,9 +130,8 @@ describe("Dispatch", () => { }); it("tells the router again what it hosts, on the new connection", async () => { - const agent = api.agent("stream-support"); - agent.tools.register({ name: "lookup", description: "Look up", run: () => "" }); - const dispatch = new Dispatch({ client: api }).host(agent); + api.tools.register({ name: "lookup", description: "Look up", run: () => "" }); + const dispatch = new Dispatch({ client: api }).host(api); const running = dispatch.run(); const first = await router.socket(); @@ -143,7 +141,7 @@ describe("Dispatch", () => { const offered = await second.next(); assert.equal(offered["type"], "host_tools"); - assert.equal(offered["agent_id"], "stream-support"); + assert.equal(offered["every_agent"], true); dispatch.stop(); await running; }); @@ -454,14 +452,13 @@ describe("Dispatch", () => { }); it("declares the tools it hosts once the router is listening, and runs the calls sent down", async () => { - const agent = api.agent("stream-support"); - agent.tools.register<{ sdk: string }>({ + api.tools.register<{ sdk: string }>({ name: "investigate_sdk", description: "Read SDK source", parameters: { type: "object", properties: { sdk: { type: "string" } } }, run: ({ sdk }) => `read ${sdk}`, }); - const dispatch = new Dispatch({ client: api }).host(agent, { + const dispatch = new Dispatch({ client: api }).host(api, { toolTimeoutMs: 60_000, }); @@ -471,7 +468,7 @@ describe("Dispatch", () => { assert.deepEqual(await connection.next(), { type: "host_tools", - agent_id: "stream-support", + every_agent: true, tools: [ { name: "investigate_sdk", @@ -482,7 +479,7 @@ describe("Dispatch", () => { timeout_ms: 60_000, }); - connection.send({ type: "hosting", agent_id: "stream-support", tools: ["investigate_sdk"] }); + connection.send({ type: "hosting", every_agent: true, tools: ["investigate_sdk"] }); connection.send({ type: "tool_call", id: "call_1", @@ -502,9 +499,8 @@ describe("Dispatch", () => { }); it("declares its tools again every time the router says it is listening", async () => { - const agent = api.agent("stream-support"); - agent.tools.register({ name: "lookup", description: "Look up", run: () => "" }); - const dispatch = new Dispatch({ client: api }).host(agent); + api.tools.register({ name: "lookup", description: "Look up", run: () => "" }); + const dispatch = new Dispatch({ client: api }).host(api); const running = dispatch.run(); const connection = await router.socket(); @@ -523,13 +519,12 @@ describe("Dispatch", () => { it("keeps reading while a hosted tool runs, and counts it as work in flight", async () => { const held: (() => void)[] = []; - const agent = api.agent("stream-support"); - agent.tools.register({ + api.tools.register({ name: "slow", description: "Takes a while", run: () => new Promise((resolve) => held.push(() => resolve("done"))), }); - const dispatch = new Dispatch({ client: api }).host(agent); + const dispatch = new Dispatch({ client: api }).host(api); const running = dispatch.run(); const connection = await router.socket(); @@ -552,15 +547,14 @@ describe("Dispatch", () => { }); it("tells the router a hosted tool failed, since a model is waiting on it", async () => { - const agent = api.agent("stream-support"); - agent.tools.register({ + api.tools.register({ name: "investigate_sdk", description: "Read SDK source", run: () => { throw new Error("the checkout is missing"); }, }); - const dispatch = new Dispatch({ client: api }).host(agent); + const dispatch = new Dispatch({ client: api }).host(api); const running = dispatch.run(); const connection = await router.socket(); @@ -585,8 +579,7 @@ describe("Dispatch", () => { }); it("declares a hosted tool's policies", async () => { - const agent = api.agent("travel"); - agent.tools.register({ + api.tools.register({ name: "book_flight", description: "Book a flight", parameters: { type: "object", properties: { flight_id: { type: "string" } } }, @@ -595,7 +588,7 @@ describe("Dispatch", () => { timeoutMs: 90_000, run: () => "booked", }); - const dispatch = new Dispatch({ client: api }).host(agent); + const dispatch = new Dispatch({ client: api }).host(api); const running = dispatch.run(); const connection = await router.socket(); @@ -611,15 +604,14 @@ describe("Dispatch", () => { }); it("stops a hosted call the router cancels, and answers nothing for it", async () => { - const agent = api.agent("support"); let release = (): void => undefined; - agent.tools.register({ + api.tools.register({ name: "investigate", description: "Investigate for a long time", run: () => new Promise((resolve) => (release = () => resolve("found it"))), }); - agent.tools.register({ name: "ping", description: "Answer at once", run: () => "pong" }); - const dispatch = new Dispatch({ client: api }).host(agent); + api.tools.register({ name: "ping", description: "Answer at once", run: () => "pong" }); + const dispatch = new Dispatch({ client: api }).host(api); const running = dispatch.run(); const connection = await router.socket(); @@ -639,10 +631,9 @@ describe("Dispatch", () => { await running; }); - it("hands a hosted function its call, and sends how it is going", async () => { - const agent = api.agent("support"); + it("hands a hosted function its call", async () => { let seen: { id: string; sessionId: string; turnId: string; user: string } | undefined; - agent.tools.register({ + api.tools.register({ name: "investigate", description: "Investigate", run: (_input, toolCallContext) => { @@ -652,11 +643,10 @@ describe("Dispatch", () => { turnId: toolCallContext.turnId, user: toolCallContext.user.id, }; - toolCallContext.update("reading the logs"); return "found it"; }, }); - const dispatch = new Dispatch({ client: api }).host(agent); + const dispatch = new Dispatch({ client: api }).host(api); const running = dispatch.run(); const connection = await router.socket(); @@ -669,40 +659,50 @@ describe("Dispatch", () => { user: { id: "sam" }, }); - assert.deepEqual(await connection.next(), { - type: "tool_update", - tool_call_id: "call_1", - text: "reading the logs", - verbatim: false, - }); assert.equal((await connection.next())["output"], "found it"); assert.deepEqual(seen, { id: "call_1", sessionId: "s1", turnId: "turn_3", user: "sam" }); dispatch.stop(); await running; }); - it("stops waiting when the router refuses its tools, saying for which agent and why", async () => { - const agent = api.agent("stream-support"); - agent.tools.register({ name: "lookup", description: "Look up", run: () => "" }); - const dispatch = new Dispatch({ client: api }).host(agent); + it("stops waiting when the router refuses its tools, saying why", async () => { + api.tools.register({ name: "lookup", description: "Look up", run: () => "" }); + const dispatch = new Dispatch({ client: api }).host(api); const running = dispatch.run(); const connection = await router.socket(); - connection.send({ - type: "hosting_refused", - agent_id: "stream-support", - reason: "hosting no tools is not hosting", - }); + connection.send({ type: "hosting_refused", reason: "hosting no tools is not hosting" }); await assert.rejects(running, (error: unknown) => { assert.ok(error instanceof HostingRefusedError); - assert.equal(error.agentId, "stream-support"); assert.equal(error.reason, "hosting no tools is not hosting"); - assert.match(error.message, /stream-support.*hosting no tools is not hosting/); + assert.match(error.message, /hosting no tools is not hosting/); return true; }); }); + it("runs the client's tools for every agent with runDispatch, until its signal aborts", async () => { + api.tools.register({ name: "lookup", description: "Look up", run: () => "found" }); + const stop = new AbortController(); + + const running = api.runDispatch(stop.signal); + const connection = await router.socket(); + assert.equal(connection.query.get("handles"), ""); + connection.send({ type: "ready", worker_id: "worker_1" }); + const offered = await connection.next(); + assert.equal(offered["type"], "host_tools"); + assert.equal(offered["every_agent"], true); + assert.deepEqual( + (offered["tools"] as Record[]).map((tool) => tool["name"]), + ["lookup"], + ); + + connection.send({ type: "tool_call", id: "call_1", session_id: "s1", name: "lookup", arguments: "{}" }); + assert.equal((await connection.next())["output"], "found"); + stop.abort(); + await running; + }); + it("answers the second message on a channel from the session that answered the first", async () => { router.serve("POST", "/v1/agents/sessions", { status: 201, diff --git a/sdks/js/tests/hardening.test.ts b/sdks/js/tests/hardening.test.ts index 9943e9489b..e24c90c600 100644 --- a/sdks/js/tests/hardening.test.ts +++ b/sdks/js/tests/hardening.test.ts @@ -114,9 +114,9 @@ describe("production hardening", () => { assert.equal(session.state.get().kind, "ended"); }); - it("refuses two hosted agents whose tool names would select the wrong implementation", () => { - const first = api.agent("first"); - const second = api.agent("second"); + it("refuses two hosted clients whose tool names would select the wrong implementation", () => { + const first = new Client({ url: router.url, customerId: "local" }); + const second = new Client({ url: router.url, customerId: "local" }); first.tools.register({ name: "lookup", description: "First lookup", run: () => "first" }); second.tools.register({ name: "lookup", description: "Second lookup", run: () => "second" }); const worker = new Dispatch({ client: api }).host(first); @@ -124,8 +124,8 @@ describe("production hardening", () => { }); it("also refuses hosted tool collisions introduced after hosting", async () => { - const first = api.agent("first"); - const second = api.agent("second"); + const first = new Client({ url: router.url, customerId: "local" }); + const second = new Client({ url: router.url, customerId: "local" }); first.tools.register({ name: "lookup", description: "First lookup", run: () => "first" }); const worker = new Dispatch({ client: api }).host(first).host(second); second.tools.register({ name: "lookup", description: "Second lookup", run: () => "second" }); @@ -133,9 +133,8 @@ describe("production hardening", () => { }); it("finishes a pending tool hold when the worker is aborted", async () => { - const agent = api.agent("support"); let aborted: boolean | undefined; - agent.tools.register({ + api.tools.register({ name: "write", description: "Write after holding the call", run: async (_input, context) => { @@ -147,7 +146,7 @@ describe("production hardening", () => { }, }); const stop = new AbortController(); - const worker = new Dispatch({ client: api }).host(agent); + const worker = new Dispatch({ client: api }).host(api); const running = worker.run(stop.signal); const connection = await router.socket(); connection.send({ type: "ready" }); diff --git a/sdks/js/tests/session.test.ts b/sdks/js/tests/session.test.ts index 20d620c456..804a716c1d 100644 --- a/sdks/js/tests/session.test.ts +++ b/sdks/js/tests/session.test.ts @@ -324,15 +324,13 @@ describe("Session", () => { await session.close(); }); - it("hands a function the call it runs for, and sends how it is going", async () => { + it("hands a function the call it runs for", async () => { let seen: ToolCallContext | undefined; const tools = new Tools().register({ name: "search", description: "Search flights", run: (_input, toolCallContext) => { seen = toolCallContext; - toolCallContext.update("checking three airlines"); - toolCallContext.say("Still looking, one moment."); return "found two"; }, }); @@ -349,18 +347,6 @@ describe("Session", () => { approved: true, }); - assert.deepEqual(await connection.next(), { - type: "tool_update", - tool_call_id: "call_1", - text: "checking three airlines", - verbatim: false, - }); - assert.deepEqual(await connection.next(), { - type: "tool_update", - tool_call_id: "call_1", - text: "Still looking, one moment.", - verbatim: true, - }); assert.equal((await connection.next())["output"], "found two"); assert.equal(seen?.id, "call_1"); assert.equal(seen?.sessionId, "sess_1", "the session the call came over"); diff --git a/sdks/js/tests/tool-calls.test.ts b/sdks/js/tests/tool-calls.test.ts new file mode 100644 index 0000000000..ac5d57eb3f --- /dev/null +++ b/sdks/js/tests/tool-calls.test.ts @@ -0,0 +1,139 @@ +import assert from "node:assert/strict"; +import { afterEach, beforeEach, describe, it } from "node:test"; + +import { createElement } from "react"; +import { renderToString } from "react-dom/server"; + +import { Client, Session, ToolCalls, type SessionEvent } from "../src/index.js"; +import { useToolCall, useToolCalls } from "../src/react.js"; +import { TestRouter, type Connection } from "./router.js"; + +const started = { + type: "tool_call.started", + tool_call_id: "call-1", + tool: "book_flight", + turn_id: "turn-1", + status: "waiting", + started_at: "2026-11-18T12:00:00Z", + duration_ms: 0, +}; + +describe("ToolCalls", () => { + it("keeps the last state of each call, in the order they started", () => { + const calls = new ToolCalls(); + calls.observe(started); + calls.observe({ ...started, tool_call_id: "call-2", tool: "search_flights", status: "in_progress" }); + calls.observe({ + ...started, + type: "tool_call.finished", + status: "success", + duration_ms: 1200, + approved_at: "2026-11-18T12:00:00.400Z", + cursor: "s.4", + }); + + assert.deepEqual([...calls.get().keys()], ["call-1", "call-2"]); + assert.deepEqual(calls.get().get("call-1"), { + type: "tool_call.finished", + tool_call_id: "call-1", + tool: "book_flight", + turn_id: "turn-1", + status: "success", + started_at: "2026-11-18T12:00:00Z", + duration_ms: 1200, + approved_at: "2026-11-18T12:00:00.400Z", + }); + assert.equal(calls.get().get("call-2")?.status, "in_progress"); + }); + + it("leaves the map alone for frames about no call's state", () => { + const calls = new ToolCalls(); + calls.observe(started); + const before = calls.get(); + + calls.observe({ type: "tool_started", tool_call_id: "call-9", tool: "x" }); + calls.observe({ type: "tool_call.progress", status: "in_progress" }); + + assert.equal(calls.get(), before, "the same map, not an equal one"); + }); +}); + +describe("session.toolCalls and session.on", () => { + let router: TestRouter; + let api: Client; + + beforeEach(async () => { + router = await TestRouter.start(); + api = new Client({ url: router.url, customerId: "local" }); + router.serve("POST", "/v1/agents/sessions", { + status: 201, + body: { + id: "sess_1", + call_id: "sess_1", + call_type: "agent", + user_id: "john", + agent_id: "john", + state: "live", + created_at: "2026-01-01T00:00:00Z", + }, + }); + }); + + afterEach(async () => { + await router.stop(); + }); + + async function opened(): Promise<[Session, Connection]> { + const opening = Session.open(api, { start_voice: true }); + const connection = await router.socket(); + return [await opening, connection]; + } + + /** Reads events until one of kind arrives. */ + async function until(session: Session, kind: string): Promise { + for await (const event of session.events()) { + if (event.kind === kind) { + return; + } + } + } + + it("keeps each call's latest state, and a handler hears its kind until it is turned off", async () => { + const [session, connection] = await opened(); + const heard: string[] = []; + let finished = 0; + const off = session.on("tool_call.progress", (event: SessionEvent) => { + heard.push(String(event.frame["progress"])); + off(); + }); + session.on("tool_call.finished", () => finished++); + + connection.send({ ...started, status: "in_progress" }); + connection.send({ ...started, type: "tool_call.progress", status: "in_progress", progress: "Searching for flights" }); + connection.send({ ...started, type: "tool_call.progress", status: "in_progress", progress: "Comparing prices" }); + connection.send({ ...started, type: "tool_call.finished", status: "failed", error: "the airline is down" }); + await until(session, "tool_call.finished"); + + assert.deepEqual(heard, ["Searching for flights"]); + assert.equal(finished, 1); + assert.equal(session.toolCall("call-1")?.status, "failed"); + assert.equal(session.toolCall("call-1")?.error, "the airline is down"); + assert.equal(session.toolCall("call-1")?.progress, undefined, "the finished frame carries the whole state"); + await session.close(); + }); + + it("renders the calls through the React hooks", async () => { + const [session, connection] = await opened(); + connection.send({ ...started, type: "tool_call.progress", status: "in_progress", progress: "Booking" }); + await until(session, "tool_call.progress"); + + function Calls() { + const calls = useToolCalls(session); + const call = useToolCall(session, "call-1"); + return createElement("p", null, `${calls.size} ${call?.status} ${call?.progress}`); + } + + assert.equal(renderToString(createElement(Calls)), "

1 in_progress Booking

"); + await session.close(); + }); +}); diff --git a/sdks/js/tests/tools.test.ts b/sdks/js/tests/tools.test.ts index 53f8dd8ceb..218eb3f366 100644 --- a/sdks/js/tests/tools.test.ts +++ b/sdks/js/tests/tools.test.ts @@ -227,8 +227,6 @@ describe("render", () => { description: "Who asked", run: async (_input, toolCallContext) => { seen = toolCallContext; - toolCallContext.update("checking"); - toolCallContext.say("One moment."); await toolCallContext.disallowInterruptions(); return "kept"; }, @@ -272,22 +270,10 @@ describe("render", () => { await assert.rejects(answer, /slow was cancelled/); }); - it("reports how a call is going on its line, and a call stopped before it was held", async () => { - const sent: [string, string, boolean][] = []; - const line = (holds: boolean) => ({ - update: (id: string, text: string, verbatim: boolean) => void sent.push([id, text, verbatim]), - hold: async () => holds, - }); - const toolCallContext = new ToolCallContext({ id: "call_1" }, line(true)); - - toolCallContext.update("checking"); - toolCallContext.say("One moment."); - await toolCallContext.disallowInterruptions(); + it("holds a call on its line, and rejects for a call stopped before it was held", async () => { + const line = (holds: boolean) => ({ hold: async () => holds }); - assert.deepEqual(sent, [ - ["call_1", "checking", false], - ["call_1", "One moment.", true], - ]); + await new ToolCallContext({ id: "call_1" }, line(true)).disallowInterruptions(); await assert.rejects( new ToolCallContext({ id: "call_2" }, line(false)).disallowInterruptions(), CallNotHeldError, diff --git a/sdks/js/tsconfig.json b/sdks/js/tsconfig.json index d6e969d5ce..869604d1f3 100644 --- a/sdks/js/tsconfig.json +++ b/sdks/js/tsconfig.json @@ -21,6 +21,7 @@ "@stream-io/stream-agents": ["./src/index.ts"], "@stream-io/stream-agents/chat": ["./src/chat.ts"], "@stream-io/stream-agents/node": ["./src/node.ts"], + "@stream-io/stream-agents/react": ["./src/react.ts"], "@stream-io/stream-agents/video": ["./src/video.ts"] } }, diff --git a/sdks/kotlin/core/src/main/kotlin/io/getstream/agents/android/core/AgentEvent.kt b/sdks/kotlin/core/src/main/kotlin/io/getstream/agents/android/core/AgentEvent.kt index cfce5fbc22..b7eb5d1ed3 100644 --- a/sdks/kotlin/core/src/main/kotlin/io/getstream/agents/android/core/AgentEvent.kt +++ b/sdks/kotlin/core/src/main/kotlin/io/getstream/agents/android/core/AgentEvent.kt @@ -7,8 +7,10 @@ import kotlinx.serialization.json.JsonElement import kotlinx.serialization.json.JsonNull import kotlinx.serialization.json.JsonObject import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.booleanOrNull import kotlinx.serialization.json.contentOrNull import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.longOrNull /** Who said something. */ public data class Participant( @@ -67,8 +69,33 @@ public data class AgentEvent( arguments = string("arguments"), requestId = string("request_id"), turnId = turnId, + run = (this["run"] as? JsonPrimitive)?.booleanOrNull ?: true, + approved = (this["approved"] as? JsonPrimitive)?.booleanOrNull ?: false, ) + /** + * The call's whole state on a `tool_call.started`, `tool_call.progress` or + * `tool_call.finished`, or null for any other event. + */ + public val toolCallState: ToolCallState? + get() { + if (kind != Kind.ToolCallStarted && kind != Kind.ToolCallProgress && kind != Kind.ToolCallFinished) return null + val id = string("tool_call_id").ifEmpty { return null } + return ToolCallState( + id = id, + tool = string("tool"), + turnId = turnId, + status = ToolCallState.Status.entries.firstOrNull { it.wire == string("status") } + ?: ToolCallState.Status.Unknown, + progress = string("progress"), + startedAt = instantOrNull(string("started_at")), + durationMs = (this["duration_ms"] as? JsonPrimitive)?.longOrNull ?: 0, + approvedAt = instantOrNull(string("approved_at")), + rejectedAt = instantOrNull(string("rejected_at")), + error = string("error"), + ) + } + /** * A connector tool call the provider refused because the person's own connection lacks * access it asked for, and the step-up consent the router began for it, or null when this @@ -114,8 +141,13 @@ public data class AgentEvent( TaskCancelled("task_cancelled"), ToolCall("tool_call"), ToolCancel("tool_cancel"), + ToolHeld("tool_held"), ToolStarted("tool_started"), + ToolUpdated("tool_updated"), ToolRan("tool_ran"), + ToolCallStarted("tool_call.started"), + ToolCallProgress("tool_call.progress"), + ToolCallFinished("tool_call.finished"), Transferred("transferred"), Pressed("pressed"), LookedUp("looked_up"), @@ -141,6 +173,10 @@ public data class AgentEvent( /** Repeated on the answer, so a result cannot be adopted by another request or turn. */ val requestId: String, val turnId: String, + /** Whether this device is the one to run it: every watcher is sent every call. */ + val run: Boolean = true, + /** Whether the person already said yes to it, on a voice call, so nobody asks again. */ + val approved: Boolean = false, ) { /** The arguments decoded, or an empty object if the model wrote something else. */ public val argumentValues: JsonObject @@ -153,6 +189,41 @@ public data class AgentEvent( } } + /** One tool call as the session last reported it, whichever process runs it. */ + public data class ToolCallState( + /** The call's `tool_call_id`. */ + val id: String, + val tool: String, + val turnId: String, + val status: Status, + /** The last thing said about the call while it runs, such as "Searching for flights", or the question it waits on. */ + val progress: String, + val startedAt: Instant?, + /** From [startedAt] to this report; once finished, how long the call took. */ + val durationMs: Long, + /** When a person allowed the call. */ + val approvedAt: Instant?, + /** When a person declined the call, or its wait for them passed. */ + val rejectedAt: Instant?, + /** Why a failed call failed, or what a declined one shows. */ + val error: String, + ) { + /** Where a call stands. [Unknown] is a status the router added after this SDK shipped. */ + public enum class Status(public val wire: String) { + InProgress("in_progress"), + /** Held for a person's approval. */ + Waiting("waiting"), + Success("success"), + Failed("failed"), + Cancelled("cancelled"), + Unknown(""), + ; + + /** Whether the call is over. */ + public val isFinished: Boolean get() = this == Success || this == Failed || this == Cancelled + } + } + /** The access a connector tool call needs, and where the person grants it. */ public data class ConnectorScopeRequired( /** The binding's alias. */ diff --git a/sdks/kotlin/core/src/main/kotlin/io/getstream/agents/android/core/AgentSession.kt b/sdks/kotlin/core/src/main/kotlin/io/getstream/agents/android/core/AgentSession.kt index 80469492f9..55be08a43e 100644 --- a/sdks/kotlin/core/src/main/kotlin/io/getstream/agents/android/core/AgentSession.kt +++ b/sdks/kotlin/core/src/main/kotlin/io/getstream/agents/android/core/AgentSession.kt @@ -22,6 +22,9 @@ import kotlinx.coroutines.flow.MutableStateFlow import kotlinx.coroutines.flow.StateFlow import kotlinx.coroutines.flow.asSharedFlow import kotlinx.coroutines.flow.asStateFlow +import kotlinx.coroutines.flow.distinctUntilChanged +import kotlinx.coroutines.flow.filter +import kotlinx.coroutines.flow.map import kotlinx.coroutines.flow.update import kotlinx.coroutines.launch import kotlinx.coroutines.sync.Mutex @@ -80,6 +83,7 @@ public class AgentSession internal constructor( private val connected = MutableStateFlow(false) private val stoppedBy = MutableStateFlow(null) private val held = MutableStateFlow>(emptyList()) + private val calls = MutableStateFlow>(emptyMap()) private val broadcast = MutableSharedFlow( extraBufferCapacity = EVENT_BUFFER, onBufferOverflow = BufferOverflow.DROP_OLDEST, @@ -107,6 +111,25 @@ public class AgentSession internal constructor( /** Calls of tools that ask first, waiting for the person to answer with [decide], oldest first. */ public val approvals: StateFlow> = held.asStateFlow() + /** + * Every tool call of the session by its id, as it was last reported: running, waiting for a + * person's approval, or how it ended, whichever process runs it. Finished calls stay. + */ + public val toolCalls: StateFlow> = calls.asStateFlow() + + /** One tool call as it was last reported, or null before the session said anything of it. */ + public fun toolCall(id: String): Flow = calls.map { it[id] }.distinctUntilChanged() + + /** + * Calls [handler] with every event of [kind], such as [AgentEvent.Kind.ToolCallFinished], on + * the session's scope until the returned job is cancelled or the session closes. It is + * subscribed before this returns, so it misses nothing sent after. + */ + public fun on(kind: AgentEvent.Kind, handler: suspend (AgentEvent) -> Unit): Job = + scope.launch(start = CoroutineStart.UNDISPATCHED) { + broadcast.filter { it.kind == kind }.collect { handler(it) } + } + /** * The session's turns: asking, reading back and rewinding. What is asked here shows in * [conversation] at once, and the reply streams into it. @@ -356,6 +379,7 @@ public class AgentSession internal constructor( private fun apply(event: AgentEvent) { state.update { it.reduce(event) } + event.toolCallState?.let { call -> calls.update { it + (call.id to call) } } when (event.kind) { AgentEvent.Kind.ToolCall -> event.toolCall?.let(::answer) AgentEvent.Kind.ToolCancel -> { @@ -382,12 +406,13 @@ public class AgentSession internal constructor( // tool this device never declared would resolve it before its owner could. The request // for frames is this device's only while its camera is on; otherwise a video worker in // the backend may own it. + if (!call.run) return if (call.name == CameraFrames.TOOL_NAME) { cameraFrames?.let { return show(it, call) } } val tool = tools[call.name] ?: return val approval = tool.approval - if (approval != null) { + if (approval != null && !call.approved) { held.update { it + ToolApprovalRequest(call, approval, tool) } } else { run(call, tool) diff --git a/sdks/kotlin/core/src/test/kotlin/io/getstream/agents/android/core/AgentSessionTest.kt b/sdks/kotlin/core/src/test/kotlin/io/getstream/agents/android/core/AgentSessionTest.kt index d1b7f48410..c5f19c290a 100644 --- a/sdks/kotlin/core/src/test/kotlin/io/getstream/agents/android/core/AgentSessionTest.kt +++ b/sdks/kotlin/core/src/test/kotlin/io/getstream/agents/android/core/AgentSessionTest.kt @@ -220,6 +220,55 @@ class AgentSessionTest { chat.close() } + @Test + fun `a call meant for another watcher is not run, and one already approved is not asked again`() = runBlocking { + val lookup = AgentTool("lookup_order", "Look up an order.") { "found" } + val refund = AgentTool("issue_refund", "Refund an order.", approval = AgentTool.Approval("Refund it?")) { "refunded" } + val chat = agents.chat(SessionOptions(tools = listOf(lookup, refund))) + + router.script.send(Script.Send("""{"type":"tool_call","id":"elsewhere","name":"lookup_order","arguments":"{}","run":false}""")) + router.script.send(Script.Send("""{"type":"tool_call","id":"approved","name":"issue_refund","arguments":"{}","approved":true}""")) + router.script.send(Script.Send("""{"type":"tool_call","id":"here","name":"lookup_order","arguments":"{}"}""")) + + val answered = setOf(router.nextSent(), router.nextSent()) + .map { Json.parseToJsonElement(it).jsonObject["tool_call_id"]?.jsonPrimitive?.content } + .toSet() + assertEquals(setOf("approved", "here"), answered) + assertTrue(chat.approvals.value.isEmpty()) + chat.close() + } + + @Test + fun `the session keeps each call's last state, and a handler hears its kind until cancelled`() = runBlocking { + val chat = agents.chat() + val progress = CopyOnWriteArrayList() + val finished = CopyOnWriteArrayList() + lateinit var first: kotlinx.coroutines.Job + first = chat.on(AgentEvent.Kind.ToolCallProgress) { + progress += it.string("progress") + first.cancel() + } + chat.on(AgentEvent.Kind.ToolCallFinished) { finished += it.string("tool_call_id") } + val call = """"tool_call_id":"c1","tool":"search_flights","turn_id":"t1","started_at":"2026-11-18T12:00:00Z"""" + + router.script.send(Script.Send("""{"type":"tool_call.started",$call,"status":"waiting","duration_ms":0}""")) + router.script.send(Script.Send("""{"type":"tool_call.progress",$call,"status":"in_progress","duration_ms":100,"progress":"Searching for flights","approved_at":"2026-11-18T12:00:00.1Z"}""")) + router.script.send(Script.Send("""{"type":"tool_call.progress",$call,"status":"in_progress","duration_ms":200,"progress":"Comparing prices","approved_at":"2026-11-18T12:00:00.1Z"}""")) + router.script.send(Script.Send("""{"type":"tool_call.finished",$call,"status":"success","duration_ms":300,"approved_at":"2026-11-18T12:00:00.1Z"}""")) + until { chat.toolCalls.value["c1"]?.status?.isFinished == true } + + val last = chat.toolCalls.value.getValue("c1") + assertEquals(AgentEvent.ToolCallState.Status.Success, last.status) + assertEquals(300, last.durationMs) + assertEquals("", last.progress, "the finished frame carries the whole state") + assertEquals(Instant.parse("2026-11-18T12:00:00.1Z"), last.approvedAt) + assertEquals(last, chat.toolCall("c1").first()) + until { finished.isNotEmpty() } + assertEquals(listOf("Searching for flights"), progress.toList()) + assertEquals(listOf("c1"), finished.toList()) + chat.close() + } + @Test fun `a request for frames with no camera on is left for the video worker that owns it`() = runBlocking { val voice = agents.chat() diff --git a/sdks/php/README.md b/sdks/php/README.md index 4249202f26..1c374fb96f 100644 --- a/sdks/php/README.md +++ b/sdks/php/README.md @@ -235,7 +235,6 @@ $agent->tools->register('book_flight', 'Book a flight for the caller', [ 'type' => 'object', 'properties' => ['flight_id' => ['type' => 'string']], ], function (ToolCallContext $toolCallContext, array $args): string { - $toolCallContext->update('checking seats'); $toolCallContext->disallowInterruptions(); return $bookings->book($args['flight_id'], user: $toolCallContext->user->id, key: $toolCallContext->id); }, timeout: 20, approval: 'Book flight {flight_id}?'); @@ -254,9 +253,7 @@ $agent->tools->register('book_flight', 'Book a flight for the caller', [ `$toolCallContext->id` is the id the model gave the call, which a call sent again keeps, so it is the key to make a side effect happen once; `sessionId`, `turnId`, `user` and `conversation` -say where it came from. `update($text)` says how the call is going: a voice agent tells the -caller in its own words once nobody is talking. `say($text)` has it say `$text` as written, -which a speech-to-speech model may paraphrase. `disallowInterruptions()` keeps the call going +say where it came from. `disallowInterruptions()` keeps the call going from then on, for a write that must not stop halfway, and throws `CallNotHeldException` when the router had already cancelled it. A PHP fiber cannot be stopped from outside, so a call that was cancelled or ran out of time keeps running with `$toolCallContext->cancelled()` true, and @@ -325,24 +322,25 @@ worker's server credential acting for the writer. ### Hosted tools -A worker can also run an agent's tools for every session opened under it, including sessions -opened from a browser. Hosting alone is enough to `run()`. +A worker can also run the app's tools, registered once on the client for every agent, for any +session, including sessions opened from a browser. The router offers a tool to a session only +where a tool group of its agent's agent.yaml names it. ```php -$agent = $client->agent('my-agent'); -$agent->tools->register('get_weather', 'The weather in a city', [ +$client->tools->register('get_weather', 'The weather in a city', [ 'type' => 'object', 'properties' => ['city' => ['type' => 'string']], ], fn (array $args): string => weatherIn($args['city'])); -$dispatch->host($agent, toolTimeoutMs: 0); // 0: the router waits its default 30 seconds per tool call -$dispatch->run(); +$client->runDispatch(); ``` Each call runs in its own fiber. A hosted tool takes the same `interruptible:`, `timeout:` and `approval:` (asked on a voice call), can take its call, and is cancelled when the router gives -up on it. A refusal throws `HostingRefusedException`, and -`$dispatch->hosting` lists the agent ids the router accepted. +up on it; without its own `timeout:` the router waits 30 seconds for a call. A refusal throws +`HostingRefusedException`. A worker that also answers calls or messages hosts the tools with +`$dispatch->host($client)` before `run()`, and `$dispatch->hosting` says whether the router +accepted them. ## Guests diff --git a/sdks/php/src/AgentHandle.php b/sdks/php/src/AgentHandle.php index c5abccc32d..bcbcdb3352 100644 --- a/sdks/php/src/AgentHandle.php +++ b/sdks/php/src/AgentHandle.php @@ -17,13 +17,10 @@ final readonly class AgentHandle { public Sessions $sessions; - /** Functions run for every session of this agent, whoever opened it, once a `Dispatch` hosts them. */ - public Tools $tools; public function __construct(private Client $client, public string $name) { $this->sessions = new Sessions($client, $name); - $this->tools = new Tools(); } /** diff --git a/sdks/php/src/Client.php b/sdks/php/src/Client.php index 00cc6944e6..915731c894 100644 --- a/sdks/php/src/Client.php +++ b/sdks/php/src/Client.php @@ -37,17 +37,24 @@ public Simulations $simulations; public SpendLimits $spendLimits; public Teams $teams; + /** The app's tools, registered once for every agent and run by `runDispatch()`. An agent is offered one only where a tool group of its agent.yaml names it. */ + public Tools $tools; private ClientInterface $http; private RequestFactoryInterface $requests; private StreamFactoryInterface $streams; + /** + * @param ?Tools $tools the app's tools, shared with a client this one is derived from + */ public function __construct( ?Backend $backend = null, ?ClientInterface $http = null, ?RequestFactoryInterface $requests = null, ?StreamFactoryInterface $streams = null, + ?Tools $tools = null, ) { $this->backend = $backend ?? new Backend(); + $this->tools = $tools ?? new Tools(); $this->requests = $requests ?? Http::requests(); $this->streams = $streams ?? Http::streams(); $this->http = $http ?? Http::client(); @@ -76,7 +83,7 @@ public function asGuest(GuestUser $guest): self public function withBackend(Backend $backend): self { - return new self($backend, $this->http, $this->requests, $this->streams); + return new self($backend, $this->http, $this->requests, $this->streams, $this->tools); } /** @@ -89,6 +96,23 @@ public function agent(string $name): AgentHandle return new AgentHandle($this, $name); } + /** + * Runs the app's tools for every session they are offered to, whoever opened it, until the + * socket closes or the process is signalled. A tool's own timeout is how long the router + * waits for one of its calls; without one it waits 30 seconds. A worker that also answers + * calls or messages hosts the tools with `Dispatch::host()` instead. + * + * $client = new Client(); + * $client->tools->register('lookup', 'Look up an order', $schema, $run); + * $client->runDispatch(); + * + * @throws Exception\HostingRefusedException when the router refuses the tools + */ + public function runDispatch(): void + { + (new Worker\Dispatch(client: $this))->host($this)->run(); + } + /** * Everything the backend routes, through a stored router config, which holds the target * each modality answers with. diff --git a/sdks/php/src/Exception/HostingRefusedException.php b/sdks/php/src/Exception/HostingRefusedException.php index 4f74610261..194090637f 100644 --- a/sdks/php/src/Exception/HostingRefusedException.php +++ b/sdks/php/src/Exception/HostingRefusedException.php @@ -11,8 +11,8 @@ */ final class HostingRefusedException extends RuntimeException implements VisionAgentsException { - public function __construct(public readonly string $agentId, public readonly string $reason) + public function __construct(public readonly string $reason) { - parent::__construct("the router refused to host tools for agent {$agentId}: {$reason}"); + parent::__construct("the router refused to host tools: {$reason}"); } } diff --git a/sdks/php/src/Generated/AgentConfig.php b/sdks/php/src/Generated/AgentConfig.php index 0ed892d380..0811035391 100644 --- a/sdks/php/src/Generated/AgentConfig.php +++ b/sdks/php/src/Generated/AgentConfig.php @@ -57,7 +57,9 @@ public function __construct( public ?string $syncHash = null, /** @var array|null */ public ?array $tags = null, - public ?AgentTools $tools = null, + // The functions an SDK provides that the agent may offer, in named groups. Empty when there are none, and the... + /** @var list|null */ + public ?array $tools = null, public ?string $tts = null, public ?SessionVideo $video = null, /** @var list|null */ @@ -118,7 +120,7 @@ public static function fromArray(array $data): self subagent: array_key_exists('subagent', $data) && $data['subagent'] !== null ? Json::string($data, 'subagent') : null, syncHash: array_key_exists('sync_hash', $data) && $data['sync_hash'] !== null ? Json::string($data, 'sync_hash') : null, tags: array_key_exists('tags', $data) && $data['tags'] !== null ? Json::stringMap($data, 'tags') : null, - tools: array_key_exists('tools', $data) && $data['tools'] !== null ? AgentTools::fromArray(Json::object($data, 'tools')) : null, + tools: array_key_exists('tools', $data) && $data['tools'] !== null ? array_map(ToolGroup::fromArray(...), Json::objects($data, 'tools')) : null, tts: array_key_exists('tts', $data) && $data['tts'] !== null ? Json::string($data, 'tts') : null, video: array_key_exists('video', $data) && $data['video'] !== null ? SessionVideo::fromArray(Json::object($data, 'video')) : null, visibleTools: array_key_exists('visible_tools', $data) && $data['visible_tools'] !== null ? Json::strings($data, 'visible_tools') : null, @@ -227,7 +229,7 @@ public function toArray(): array $out['tags'] = $this->tags; } if ($this->tools !== null) { - $out['tools'] = $this->tools->toArray(); + $out['tools'] = array_map(static fn (ToolGroup $each): array => $each->toArray(), $this->tools); } if ($this->tts !== null) { $out['tts'] = $this->tts; diff --git a/sdks/php/src/Generated/AgentConfigPatch.php b/sdks/php/src/Generated/AgentConfigPatch.php index 6c9ba436fa..4a22a17b0b 100644 --- a/sdks/php/src/Generated/AgentConfigPatch.php +++ b/sdks/php/src/Generated/AgentConfigPatch.php @@ -63,7 +63,9 @@ public function __construct( public ?string $subagent = null, /** @var array|null */ public ?array $tags = null, - public ?AgentTools $tools = null, + // Replaces the tool groups whole; an empty list removes them all. + /** @var list|null */ + public ?array $tools = null, public ?string $tts = null, public ?SessionVideo $video = null, // Tools whose steps end users see on a persistent conversation's replies, as tool names or path.Match pattern... @@ -124,7 +126,7 @@ public static function fromArray(array $data): self stt: array_key_exists('stt', $data) && $data['stt'] !== null ? Json::string($data, 'stt') : null, subagent: array_key_exists('subagent', $data) && $data['subagent'] !== null ? Json::string($data, 'subagent') : null, tags: array_key_exists('tags', $data) && $data['tags'] !== null ? Json::stringMap($data, 'tags') : null, - tools: array_key_exists('tools', $data) && $data['tools'] !== null ? AgentTools::fromArray(Json::object($data, 'tools')) : null, + tools: array_key_exists('tools', $data) && $data['tools'] !== null ? array_map(ToolGroup::fromArray(...), Json::objects($data, 'tools')) : null, tts: array_key_exists('tts', $data) && $data['tts'] !== null ? Json::string($data, 'tts') : null, video: array_key_exists('video', $data) && $data['video'] !== null ? SessionVideo::fromArray(Json::object($data, 'video')) : null, visibleTools: array_key_exists('visible_tools', $data) && $data['visible_tools'] !== null ? Json::strings($data, 'visible_tools') : null, @@ -239,7 +241,7 @@ public function toArray(): array $out['tags'] = $this->tags; } if ($this->tools !== null) { - $out['tools'] = $this->tools->toArray(); + $out['tools'] = array_map(static fn (ToolGroup $each): array => $each->toArray(), $this->tools); } if ($this->tts !== null) { $out['tts'] = $this->tts; diff --git a/sdks/php/src/Generated/AgentConfigRequest.php b/sdks/php/src/Generated/AgentConfigRequest.php index 65b7089266..30b2766063 100644 --- a/sdks/php/src/Generated/AgentConfigRequest.php +++ b/sdks/php/src/Generated/AgentConfigRequest.php @@ -72,8 +72,9 @@ public function __construct( // Cost labels, carried onto every request a session using it makes. The team tag is also the team those sessi... /** @var array|null */ public ?array $tags = null, - // How the agent is offered its connector tools. Left out on an update, the stored settings stay. - public ?AgentTools $tools = null, + // The functions an SDK provides that the agent may offer, in named groups, each with how it is loaded. A func... + /** @var list|null */ + public ?array $tools = null, public ?string $tts = null, public ?SessionVideo $video = null, // Tools whose steps end users see on a persistent conversation's replies, as tool names or path.Match pattern... @@ -134,7 +135,7 @@ public static function fromArray(array $data): self stt: array_key_exists('stt', $data) && $data['stt'] !== null ? Json::string($data, 'stt') : null, subagent: array_key_exists('subagent', $data) && $data['subagent'] !== null ? Json::string($data, 'subagent') : null, tags: array_key_exists('tags', $data) && $data['tags'] !== null ? Json::stringMap($data, 'tags') : null, - tools: array_key_exists('tools', $data) && $data['tools'] !== null ? AgentTools::fromArray(Json::object($data, 'tools')) : null, + tools: array_key_exists('tools', $data) && $data['tools'] !== null ? array_map(ToolGroup::fromArray(...), Json::objects($data, 'tools')) : null, tts: array_key_exists('tts', $data) && $data['tts'] !== null ? Json::string($data, 'tts') : null, video: array_key_exists('video', $data) && $data['video'] !== null ? SessionVideo::fromArray(Json::object($data, 'video')) : null, visibleTools: array_key_exists('visible_tools', $data) && $data['visible_tools'] !== null ? Json::strings($data, 'visible_tools') : null, @@ -247,7 +248,7 @@ public function toArray(): array $out['tags'] = $this->tags; } if ($this->tools !== null) { - $out['tools'] = $this->tools->toArray(); + $out['tools'] = array_map(static fn (ToolGroup $each): array => $each->toArray(), $this->tools); } if ($this->tts !== null) { $out['tts'] = $this->tts; diff --git a/sdks/php/src/Generated/AgentConnectorBinding.php b/sdks/php/src/Generated/AgentConnectorBinding.php index a70745ebd6..5f4ca550cf 100644 --- a/sdks/php/src/Generated/AgentConnectorBinding.php +++ b/sdks/php/src/Generated/AgentConnectorBinding.php @@ -43,6 +43,10 @@ public function __construct( // Names or path.Match patterns of the tools this binding offers, such as search or linear_*. Set, they are th... /** @var list|null */ public ?array $toolPatterns = null, + // How the binding's tools are offered to the model. Omitted is full. + public ToolLoading|string|null $loading = null, + // How long the agent's sessions keep what the binding's tools were given and gave back on the turns they reco... + public ?string $outputRetention = null, ) { } @@ -63,6 +67,8 @@ public static function fromArray(array $data): self sandbox: array_key_exists('sandbox', $data) && $data['sandbox'] !== null ? Json::bool($data, 'sandbox') : null, timeoutMs: array_key_exists('timeout_ms', $data) && $data['timeout_ms'] !== null ? Json::int($data, 'timeout_ms') : null, toolPatterns: array_key_exists('tool_patterns', $data) && $data['tool_patterns'] !== null ? Json::strings($data, 'tool_patterns') : null, + loading: array_key_exists('loading', $data) && $data['loading'] !== null ? Json::enum($data, 'loading', ToolLoading::class) : null, + outputRetention: array_key_exists('output_retention', $data) && $data['output_retention'] !== null ? Json::string($data, 'output_retention') : null, ); } @@ -99,6 +105,12 @@ public function toArray(): array if ($this->toolPatterns !== null) { $out['tool_patterns'] = $this->toolPatterns; } + if ($this->loading !== null) { + $out['loading'] = Json::enumValue($this->loading); + } + if ($this->outputRetention !== null) { + $out['output_retention'] = $this->outputRetention; + } return $out; } } diff --git a/sdks/php/src/Generated/AgentStats.php b/sdks/php/src/Generated/AgentStats.php index f36bb6c52e..ad91074207 100644 --- a/sdks/php/src/Generated/AgentStats.php +++ b/sdks/php/src/Generated/AgentStats.php @@ -20,6 +20,10 @@ public function __construct( // One row per tool, most called first. /** @var list */ public array $tools, + // Turns a session took in the window, on average. A turn is a reply to the person, not a rewound one or a del... + public float $avgTurns, + // Sessions that took a turn in the window: what a tool's session_share is out of. + public int $sessions, ) { } @@ -33,6 +37,8 @@ public static function fromArray(array $data): self peakConcurrentSessions: Json::int($data, 'peak_concurrent_sessions'), toolCalls: ToolStats::fromArray(Json::object($data, 'tool_calls')), tools: array_map(ToolStats::fromArray(...), Json::objects($data, 'tools')), + avgTurns: Json::float($data, 'avg_turns'), + sessions: Json::int($data, 'sessions'), ); } @@ -48,6 +54,8 @@ public function toArray(): array $out['peak_concurrent_sessions'] = $this->peakConcurrentSessions; $out['tool_calls'] = $this->toolCalls->toArray(); $out['tools'] = array_map(static fn (ToolStats $each): array => $each->toArray(), $this->tools); + $out['avg_turns'] = $this->avgTurns; + $out['sessions'] = $this->sessions; return $out; } } diff --git a/sdks/php/src/Generated/AgentTools.php b/sdks/php/src/Generated/AgentTools.php deleted file mode 100644 index af98f623a5..0000000000 --- a/sdks/php/src/Generated/AgentTools.php +++ /dev/null @@ -1,51 +0,0 @@ - $data - */ - public static function fromArray(array $data): self - { - return new self( - outputRetention: array_key_exists('output_retention', $data) && $data['output_retention'] !== null ? Json::string($data, 'output_retention') : null, - progressive: array_key_exists('progressive', $data) && $data['progressive'] !== null ? Json::bool($data, 'progressive') : null, - ); - } - - /** - * The request body, leaving out every field that was not set. - * - * @return array - */ - public function toArray(): array - { - $out = []; - if ($this->outputRetention !== null) { - $out['output_retention'] = $this->outputRetention; - } - if ($this->progressive !== null) { - $out['progressive'] = $this->progressive; - } - return $out; - } -} diff --git a/sdks/php/src/Generated/OfferedTool.php b/sdks/php/src/Generated/OfferedTool.php index 121151f170..23def112bc 100644 --- a/sdks/php/src/Generated/OfferedTool.php +++ b/sdks/php/src/Generated/OfferedTool.php @@ -25,6 +25,8 @@ public function __construct( public ?array $parameters = null, // Where the tool came from: session, declared by whoever opened the session; worker, hosted by a worker conne... public ?string $source = null, + // How the tool is offered, as its tool group or connector binding says: full for the router's own. A deferred... + public ToolLoading|string|null $loading = null, ) { } @@ -39,6 +41,7 @@ public static function fromArray(array $data): self tokens: Json::int($data, 'tokens'), parameters: array_key_exists('parameters', $data) && $data['parameters'] !== null ? Json::object($data, 'parameters') : null, source: array_key_exists('source', $data) && $data['source'] !== null ? Json::string($data, 'source') : null, + loading: array_key_exists('loading', $data) && $data['loading'] !== null ? Json::enum($data, 'loading', ToolLoading::class) : null, ); } @@ -59,6 +62,9 @@ public function toArray(): array if ($this->source !== null) { $out['source'] = $this->source; } + if ($this->loading !== null) { + $out['loading'] = Json::enumValue($this->loading); + } return $out; } } diff --git a/sdks/php/src/Generated/SyncAgentRequest.php b/sdks/php/src/Generated/SyncAgentRequest.php index 0025ea8449..987df621d1 100644 --- a/sdks/php/src/Generated/SyncAgentRequest.php +++ b/sdks/php/src/Generated/SyncAgentRequest.php @@ -83,8 +83,9 @@ public function __construct( public ?string $subagent = null, /** @var array|null */ public ?array $tags = null, - // How connector tools are offered. A setting left out keeps what is stored. - public ?AgentTools $tools = null, + // agent.yaml's tools: the functions an SDK provides that the agent may offer, in named groups, as AgentConfig... + /** @var list|null */ + public ?array $tools = null, public ?string $tts = null, public ?SessionVideo $video = null, // The voice the agent speaks in, as TtsOptions.voice takes it: a provider's own voice id, or one of your voic... @@ -143,7 +144,7 @@ public static function fromArray(array $data): self stt: array_key_exists('stt', $data) && $data['stt'] !== null ? Json::string($data, 'stt') : null, subagent: array_key_exists('subagent', $data) && $data['subagent'] !== null ? Json::string($data, 'subagent') : null, tags: array_key_exists('tags', $data) && $data['tags'] !== null ? Json::stringMap($data, 'tags') : null, - tools: array_key_exists('tools', $data) && $data['tools'] !== null ? AgentTools::fromArray(Json::object($data, 'tools')) : null, + tools: array_key_exists('tools', $data) && $data['tools'] !== null ? array_map(ToolGroup::fromArray(...), Json::objects($data, 'tools')) : null, tts: array_key_exists('tts', $data) && $data['tts'] !== null ? Json::string($data, 'tts') : null, video: array_key_exists('video', $data) && $data['video'] !== null ? SessionVideo::fromArray(Json::object($data, 'video')) : null, voice: array_key_exists('voice', $data) && $data['voice'] !== null ? Json::string($data, 'voice') : null, @@ -264,7 +265,7 @@ public function toArray(): array $out['tags'] = $this->tags; } if ($this->tools !== null) { - $out['tools'] = $this->tools->toArray(); + $out['tools'] = array_map(static fn (ToolGroup $each): array => $each->toArray(), $this->tools); } if ($this->tts !== null) { $out['tts'] = $this->tts; diff --git a/sdks/php/src/Generated/ToolCallFrameType.php b/sdks/php/src/Generated/ToolCallFrameType.php new file mode 100644 index 0000000000..2aefbadf95 --- /dev/null +++ b/sdks/php/src/Generated/ToolCallFrameType.php @@ -0,0 +1,18 @@ + $data + */ + public static function fromArray(array $data): self + { + return new self( + durationMs: Json::int($data, 'duration_ms'), + startedAt: Json::date($data, 'started_at'), + status: Json::enum($data, 'status', ToolCallStatus::class), + tool: Json::string($data, 'tool'), + toolCallId: Json::string($data, 'tool_call_id'), + turnId: Json::string($data, 'turn_id'), + type: Json::enum($data, 'type', ToolCallFrameType::class), + approvedAt: array_key_exists('approved_at', $data) && $data['approved_at'] !== null ? Json::date($data, 'approved_at') : null, + error: array_key_exists('error', $data) && $data['error'] !== null ? Json::string($data, 'error') : null, + progress: array_key_exists('progress', $data) && $data['progress'] !== null ? Json::string($data, 'progress') : null, + rejectedAt: array_key_exists('rejected_at', $data) && $data['rejected_at'] !== null ? Json::date($data, 'rejected_at') : null, + ); + } + + /** + * The request body, leaving out every field that was not set. + * + * @return array + */ + public function toArray(): array + { + $out = []; + $out['duration_ms'] = $this->durationMs; + $out['started_at'] = Json::dateValue($this->startedAt); + $out['status'] = Json::enumValue($this->status); + $out['tool'] = $this->tool; + $out['tool_call_id'] = $this->toolCallId; + $out['turn_id'] = $this->turnId; + $out['type'] = Json::enumValue($this->type); + if ($this->approvedAt !== null) { + $out['approved_at'] = Json::dateValue($this->approvedAt); + } + if ($this->error !== null) { + $out['error'] = $this->error; + } + if ($this->progress !== null) { + $out['progress'] = $this->progress; + } + if ($this->rejectedAt !== null) { + $out['rejected_at'] = Json::dateValue($this->rejectedAt); + } + return $out; + } +} diff --git a/sdks/php/src/Generated/ToolCallStatus.php b/sdks/php/src/Generated/ToolCallStatus.php new file mode 100644 index 0000000000..f42597f0dd --- /dev/null +++ b/sdks/php/src/Generated/ToolCallStatus.php @@ -0,0 +1,21 @@ + */ + public array $tools, + public ToolLoading|string|null $loading = null, + // How long the agent's sessions keep what the group's tools were given and gave back on the turns they record... + public ?string $outputRetention = null, + ) { + } + + /** + * @param array $data + */ + public static function fromArray(array $data): self + { + return new self( + name: Json::string($data, 'name'), + tools: Json::strings($data, 'tools'), + loading: array_key_exists('loading', $data) && $data['loading'] !== null ? Json::enum($data, 'loading', ToolLoading::class) : null, + outputRetention: array_key_exists('output_retention', $data) && $data['output_retention'] !== null ? Json::string($data, 'output_retention') : null, + ); + } + + /** + * The request body, leaving out every field that was not set. + * + * @return array + */ + public function toArray(): array + { + $out = []; + $out['name'] = $this->name; + $out['tools'] = $this->tools; + if ($this->loading !== null) { + $out['loading'] = Json::enumValue($this->loading); + } + if ($this->outputRetention !== null) { + $out['output_retention'] = $this->outputRetention; + } + return $out; + } +} diff --git a/sdks/php/src/Generated/ToolLoading.php b/sdks/php/src/Generated/ToolLoading.php new file mode 100644 index 0000000000..fc56d11cd5 --- /dev/null +++ b/sdks/php/src/Generated/ToolLoading.php @@ -0,0 +1,23 @@ +failed; $out['p99_ms'] = $this->p99Ms; $out['tool'] = $this->tool; + $out['avg_turns_before_use'] = $this->avgTurnsBeforeUse; + $out['definition_tokens'] = $this->definitionTokens; + $out['loading'] = $this->loading; + $out['session_share'] = $this->sessionShare; + $out['sessions'] = $this->sessions; + $out['summary_tokens'] = $this->summaryTokens; return $out; } } diff --git a/sdks/php/src/ToolCallContext.php b/sdks/php/src/ToolCallContext.php index 704344fb77..07326f27ba 100644 --- a/sdks/php/src/ToolCallContext.php +++ b/sdks/php/src/ToolCallContext.php @@ -66,23 +66,6 @@ public static function fromFrame(array $frame, string $sessionId, Closure $send, ); } - /** - * Tells the people in the conversation how the call is going, such as "Found three - * flights, comparing prices". The agent puts it in its own words. - */ - public function update(string $text): void - { - $this->sendUpdate($text, false); - } - - /** - * `update()`, said word for word. - */ - public function say(string $text): void - { - $this->sendUpdate($text, true); - } - /** * Keeps the call going if the person talks over the agent from here on, for a call that is * past the point of taking back, such as one that has started a payment. @@ -131,11 +114,4 @@ public function stop(): void { $this->stopping->cancel(); } - - private function sendUpdate(string $text, bool $verbatim): void - { - if ($this->send !== null) { - ($this->send)(['type' => 'tool_update', 'tool_call_id' => $this->id, 'text' => $text, 'verbatim' => $verbatim]); - } - } } diff --git a/sdks/php/src/Worker/Dispatch.php b/sdks/php/src/Worker/Dispatch.php index 16f5d734c6..ddeca5b671 100644 --- a/sdks/php/src/Worker/Dispatch.php +++ b/sdks/php/src/Worker/Dispatch.php @@ -11,7 +11,6 @@ use Amp\CancelledException; use Closure; use GetStream\VisionAgents\Agent; -use GetStream\VisionAgents\AgentHandle; use GetStream\VisionAgents\AgentResponse; use GetStream\VisionAgents\Client; use GetStream\VisionAgents\Exception\ConfigurationException; @@ -54,10 +53,10 @@ final class Dispatch public readonly Client $client; /** What the router calls this connection, for matching a log line here against one there. */ public string $workerId = ''; - /** @var list the agent ids the router has said it sends this worker's tools for */ - public array $hosting = []; + /** Whether the router has said it sends this worker the calls of the tools it hosts. */ + public bool $hosting = false; - /** @var list */ + /** @var list */ private array $hosted = []; /** @var array> */ private array $running = []; @@ -127,23 +126,24 @@ public function waitForMessage(callable $handler): self } /** - * Runs the agent's tools for every session opened under it, whoever opened it. + * Runs the app's tools, `$client->tools`, for every session they are offered to, whoever + * opened it. * - * A session's own tools run in the process that opened it, which is no use to a conversation - * opened from a browser. The router offers `$agent->tools` to each session naming the agent - * and sends every call here. Call before `run()`. + * The tools are registered once on the client, for every agent: the router offers one to a + * session only where a tool group of its agent's agent.yaml names it, and sends every call + * here. `Client::runDispatch()` does this alone, for a worker with no calls or messages to + * answer. Call before `run()`. * - * $agent = $client->agent('my-agent'); - * $agent->tools->register('lookup', 'Look up an order', $schema, $run); - * (new Dispatch())->host($agent)->run(); + * $client->tools->register('lookup', 'Look up an order', $schema, $run); + * (new Dispatch(client: $client))->host($client)->run(); * * @param int $toolTimeoutMs how long the router waits for one tool call to be answered before * telling the model it failed, not how long the worker runs; 0 takes * the router's default of 30 seconds */ - public function host(AgentHandle $agent, int $toolTimeoutMs = 0): self + public function host(Client $client, int $toolTimeoutMs = 0): self { - $this->hosted[] = ['agentId' => $agent->name, 'tools' => $agent->tools, 'toolTimeoutMs' => $toolTimeoutMs]; + $this->hosted[] = ['tools' => $client->tools, 'toolTimeoutMs' => $toolTimeoutMs]; return $this; } @@ -211,7 +211,7 @@ public function run(): void } $socket = Socket::open($this->client, '/v1/dispatch', $this->waiting()); $this->socket = $socket; - $this->hosting = []; + $this->hosting = false; $this->holds = new Holds(); $watchers = [EventLoop::repeat($this->reportEvery, fn () => $this->report())]; @@ -232,9 +232,9 @@ public function run(): void 'tool_call' => $this->runHosted($frame), 'tool_held' => $this->holds->answer(self::hostedKey($frame), true), 'tool_cancel' => $this->cancelHosted(self::hostedKey($frame)), - 'hosting' => $this->hosting[] = Json::string($frame, 'agent_id'), + 'hosting' => $this->hosting = true, // A worker whose tools were refused is one nobody will call. - 'hosting_refused' => throw new HostingRefusedException(Json::string($frame, 'agent_id'), Json::string($frame, 'reason')), + 'hosting_refused' => throw new HostingRefusedException(Json::string($frame, 'reason')), 'pong' => $this->pong?->isComplete() === false ? $this->pong->complete(Json::float($frame, 'at')) : null, default => null, }; @@ -296,7 +296,7 @@ private function ready(string $workerId): void foreach ($this->hosted as $offer) { $this->tell([ 'type' => 'host_tools', - 'agent_id' => $offer['agentId'], + 'every_agent' => true, 'tools' => array_map(static fn (SessionTool $tool): array => $tool->toArray(), $offer['tools']->declared()), 'timeout_ms' => $offer['toolTimeoutMs'], ]); diff --git a/sdks/php/tests/Unit/ClientTest.php b/sdks/php/tests/Unit/ClientTest.php index 01106bd64f..9e5ca6eb7c 100644 --- a/sdks/php/tests/Unit/ClientTest.php +++ b/sdks/php/tests/Unit/ClientTest.php @@ -32,6 +32,16 @@ protected function tearDown(): void $this->router->stop(); } + public function testAClientActingForAUserSharesTheAppsTools(): void + { + $client = $this->router->client(); + $client->tools->register('weather_lookup', 'The weather somewhere', [], static fn (array $args): string => 'sunny'); + + $user = $client->asUser('ada', 'token'); + + self::assertSame(['weather_lookup'], array_map(static fn ($tool): string => $tool->name, $user->tools->declared())); + } + public function testNamingNoRouterGoesToTheHostedOneThroughTheProxy(): void { $saved = getenv(Backend::URL_ENV); diff --git a/sdks/php/tests/Unit/ToolCallTest.php b/sdks/php/tests/Unit/ToolCallTest.php index 9b4428c7d3..f4f7d43740 100644 --- a/sdks/php/tests/Unit/ToolCallTest.php +++ b/sdks/php/tests/Unit/ToolCallTest.php @@ -41,7 +41,7 @@ protected function tearDown(): void $this->server?->stop(); } - public function testAToolIsToldWhoItsCallAnswersAndCanSayHowItIsGoing(): void + public function testAToolIsToldWhoItsCallAnswers(): void { $server = $this->serve(static function (WebsocketClient $socket, LocalSocketServer $server): void { LocalSocketServer::send($socket, [ @@ -53,18 +53,12 @@ public function testAToolIsToldWhoItsCallAnswersAndCanSayHowItIsGoing(): void }); $agent = new Agent(name: 'jean', client: $server->client()); $agent->tools->register('book', 'Books a flight', [], static function (ToolCallContext $toolCallContext, array $args): string { - $toolCallContext->update('Looking at flights'); - $toolCallContext->say('One moment.'); return implode(' ', [$toolCallContext->id, $toolCallContext->sessionId, $toolCallContext->turnId, $toolCallContext->user->id, $toolCallContext->user->name, Json::string($toolCallContext->user->custom, 'tier'), $toolCallContext->conversation->cid, $toolCallContext->approved ? 'approved' : 'asked']); }); self::watch($server, $agent); - self::assertSame([ - ['type' => 'tool_update', 'tool_call_id' => 't1', 'text' => 'Looking at flights', 'verbatim' => false], - ['type' => 'tool_update', 'tool_call_id' => 't1', 'text' => 'One moment.', 'verbatim' => true], - ], $server->sent('tool_update')); self::assertSame('t1 ses_1 turn-1 u1 Ada gold messaging:c1 approved', $server->sent('tool_result')[0]['output']); } @@ -203,10 +197,10 @@ public function testAHostedToolIsDeclaredWithItsPoliciesAndAnsweredByCallId(): v self::until($socket, $server, 'tool_result'); $socket->close(); }); - $agent = $server->client()->agent('payments'); - self::payments($agent->tools); + $client = $server->client(); + self::payments($client->tools); - (new Dispatch(client: $server->client()))->host($agent)->run(); + (new Dispatch(client: $client))->host($client)->run(); $declared = []; foreach (Json::objects($server->sent('host_tools')[0], 'tools') as $tool) { @@ -233,10 +227,10 @@ public function testAHostedCallCancelledByAnInterruptionIsNotAnswered(): void delay(0.4); $socket->close(); }); - $agent = $server->client()->agent('payments'); - self::payments($agent->tools); + $client = $server->client(); + self::payments($client->tools); - (new Dispatch(client: $server->client()))->host($agent)->run(); + (new Dispatch(client: $client))->host($client)->run(); self::assertSame([['type' => 'tool_result', 'tool_call_id' => 'c2', 'id' => 'c2', 'output' => 'done']], $server->sent('tool_result')); } @@ -253,10 +247,10 @@ public function testAHostedCallHeldThroughInterruptionsIsAnswered(): void self::until($socket, $server, 'tool_result'); $socket->close(); }); - $agent = $server->client()->agent('payments'); - self::payments($agent->tools); + $client = $server->client(); + self::payments($client->tools); - (new Dispatch(client: $server->client()))->host($agent)->run(); + (new Dispatch(client: $client))->host($client)->run(); self::assertSame([['type' => 'tool_hold', 'tool_call_id' => 'c1']], $server->sent('tool_hold')); self::assertSame([['type' => 'tool_result', 'tool_call_id' => 'c1', 'id' => 'c1', 'output' => 'paid for s1']], $server->sent('tool_result')); diff --git a/sdks/php/tests/Unit/WorkerTest.php b/sdks/php/tests/Unit/WorkerTest.php index f237f63071..4129b49359 100644 --- a/sdks/php/tests/Unit/WorkerTest.php +++ b/sdks/php/tests/Unit/WorkerTest.php @@ -116,10 +116,10 @@ public function testDispatchSaysDoneWithAnErrorForAMessageItHasNoHandlerFor(): v } $socket->close(); }); - $agent = $server->client()->agent('my-agent'); - $agent->tools->register('weather_lookup', 'The weather somewhere', [], static fn (array $args): string => 'sunny'); + $client = $server->client(); + $client->tools->register('weather_lookup', 'The weather somewhere', [], static fn (array $args): string => 'sunny'); - (new Dispatch(client: $server->client()))->host($agent)->run(); + (new Dispatch(client: $client))->host($client)->run(); self::assertSame('/v1/dispatch?capacity=4&active=0&handles=', $server->handshakes[0]); self::assertSame([['type' => 'done', 'work_id' => 'wk1', 'error' => 'this worker answers no messages']], $server->sent('done')); @@ -278,7 +278,7 @@ public function testDispatchDeclaresAndAnswersHostedTools(): void $server = $this->serve(static function (WebsocketClient $socket, LocalSocketServer $server): void { LocalSocketServer::send($socket, ['type' => 'ready', 'worker_id' => 'w-1']); $server->next($socket); - LocalSocketServer::send($socket, ['type' => 'hosting', 'agent_id' => 'my-agent']); + LocalSocketServer::send($socket, ['type' => 'hosting', 'every_agent' => true]); LocalSocketServer::send($socket, ['type' => 'tool_call', 'id' => 't1', 'session_id' => 'ses_1', 'name' => 'slow', 'arguments' => '{}']); LocalSocketServer::send($socket, ['type' => 'tool_call', 'id' => 't2', 'session_id' => 'ses_1', 'name' => 'weather_lookup', 'arguments' => '{"location":"Boulder, Colorado"}']); LocalSocketServer::send($socket, ['type' => 'tool_call', 'id' => 't3', 'session_id' => 'ses_1', 'name' => 'broken', 'arguments' => '{}']); @@ -287,21 +287,21 @@ public function testDispatchDeclaresAndAnswersHostedTools(): void } $socket->close(); }); - $agent = $server->client()->agent('my-agent'); - $agent->tools + $client = $server->client(); + $client->tools ->register('weather_lookup', 'The weather somewhere', ['type' => 'object', 'properties' => ['location' => ['type' => 'string']]], static fn (array $args): array => ['location' => $args['location'], 'sky' => 'sunny']) ->register('slow', 'Takes a while', [], static function (array $args): string { delay(0.2); return 'done'; }) ->register('broken', 'Always fails', [], static fn (array $args): string => throw new RuntimeException('the weather service is down')); - $dispatch = (new Dispatch(client: $server->client()))->host($agent, toolTimeoutMs: 30000); + $dispatch = (new Dispatch(client: $client))->host($client, toolTimeoutMs: 30000); $dispatch->run(); self::assertSame([[ 'type' => 'host_tools', - 'agent_id' => 'my-agent', + 'every_agent' => true, 'tools' => [ ['description' => 'The weather somewhere', 'name' => 'weather_lookup', 'parameters' => ['type' => 'object', 'properties' => ['location' => ['type' => 'string']]]], ['description' => 'Takes a while', 'name' => 'slow'], @@ -309,7 +309,7 @@ public function testDispatchDeclaresAndAnswersHostedTools(): void ], 'timeout_ms' => 30000, ]], $server->sent('host_tools')); - self::assertSame(['my-agent'], $dispatch->hosting); + self::assertTrue($dispatch->hosting); $results = []; foreach ($server->sent('tool_result') as $result) { $results[Json::string($result, 'id')] = $result; @@ -329,9 +329,9 @@ public function testDispatchHostsAgainOnEveryConnection(): void $server->next($socket); $socket->close(); }); - $agent = $server->client()->agent('my-agent'); - $agent->tools->register('weather_lookup', 'The weather somewhere', [], static fn (array $args): string => 'sunny'); - $dispatch = (new Dispatch(client: $server->client()))->host($agent); + $client = $server->client(); + $client->tools->register('weather_lookup', 'The weather somewhere', [], static fn (array $args): string => 'sunny'); + $dispatch = (new Dispatch(client: $client))->host($client); $dispatch->run(); $dispatch->run(); @@ -344,22 +344,44 @@ public function testDispatchHostsAgainOnEveryConnection(): void public function testDispatchStopsWhenHostingIsRefused(): void { $server = $this->serve(static function (WebsocketClient $socket, LocalSocketServer $server): void { - LocalSocketServer::send($socket, ['type' => 'hosting_refused', 'agent_id' => 'my-agent', 'reason' => 'hosting no tools is not hosting']); + LocalSocketServer::send($socket, ['type' => 'hosting_refused', 'reason' => 'hosting no tools is not hosting']); while ($server->next($socket) !== null) { } }); - $dispatch = (new Dispatch(client: $server->client()))->host($server->client()->agent('my-agent')); + $client = $server->client(); + $dispatch = (new Dispatch(client: $client))->host($client); try { $dispatch->run(); self::fail('a worker nobody will call kept waiting'); } catch (HostingRefusedException $refused) { - self::assertSame('my-agent', $refused->agentId); self::assertSame('hosting no tools is not hosting', $refused->reason); - self::assertSame('the router refused to host tools for agent my-agent: hosting no tools is not hosting', $refused->getMessage()); + self::assertSame('the router refused to host tools: hosting no tools is not hosting', $refused->getMessage()); } } + public function testRunDispatchRunsTheClientsToolsForEveryAgent(): void + { + $server = $this->serve(static function (WebsocketClient $socket, LocalSocketServer $server): void { + LocalSocketServer::send($socket, ['type' => 'ready', 'worker_id' => 'w-1']); + $server->next($socket); + LocalSocketServer::send($socket, ['type' => 'tool_call', 'id' => 't1', 'session_id' => 'ses_1', 'name' => 'weather_lookup', 'arguments' => '{}']); + while ($server->sent('tool_result') === [] && $server->next($socket) !== null) { + } + $socket->close(); + }); + $client = $server->client(); + $client->tools->register('weather_lookup', 'The weather somewhere', [], static fn (array $args): string => 'sunny'); + + $client->runDispatch(); + + self::assertSame('/v1/dispatch?capacity=4&active=0&handles=', $server->handshakes[0]); + $declared = $server->sent('host_tools')[0]; + self::assertTrue($declared['every_agent']); + self::assertSame(['weather_lookup'], array_map(static fn (array $tool): mixed => $tool['name'], Json::objects($declared, 'tools'))); + self::assertSame([['type' => 'tool_result', 'tool_call_id' => 't1', 'id' => 't1', 'output' => 'sunny']], $server->sent('tool_result')); + } + public function testDispatchNeedsAHandlerOrHostedTools(): void { $this->expectException(ConfigurationException::class); diff --git a/sdks/python/vision_agents/core/llm/tool_call.py b/sdks/python/vision_agents/core/llm/tool_call.py index 2bece35063..cbf3ce50ed 100644 --- a/sdks/python/vision_agents/core/llm/tool_call.py +++ b/sdks/python/vision_agents/core/llm/tool_call.py @@ -15,12 +15,9 @@ class ToolLine(Protocol): """How a running call reaches the session it runs for. The socket that delivered the call provides one. Without one, as outside a call, - `ToolCallContext.update`, `say` and `disallow_interruptions` do nothing. + `ToolCallContext.disallow_interruptions` does nothing. """ - async def update(self, call_id: str, text: str, verbatim: bool) -> None: - """Say how the call is going, or with `verbatim`, words to say as written.""" - async def hold(self, call_id: str) -> bool: """Ask for the call to keep going when the caller talks over the agent. @@ -53,7 +50,6 @@ class ToolCallContext: @llm.register_function(description="Book a flight") async def book_flight(tool_call_context: ToolCallContext, flight_id: str) -> str: - await tool_call_context.update("checking seats") return await bookings.book(flight_id, key=tool_call_context.id) Outside a call, as under a model that does not say, it is an empty one whose methods @@ -73,25 +69,6 @@ async def book_flight(tool_call_context: ToolCallContext, flight_id: str) -> str approved: bool = False line: Optional[ToolLine] = field(default=None, repr=False, compare=False) - async def update(self, text: str) -> None: - """Say how the call is going. - - A voice agent tells the caller in its own words once nobody is talking, and a - conversation shows it on the call's step until the call ends. A newer update - replaces one not yet said, and the result replaces both. - """ - if self.line is not None and self.id: - await self.line.update(self.id, text, False) - - async def say(self, text: str) -> None: - """Have the agent say `text` as written while the call runs. - - A text conversation posts it as a message. A speech-to-speech model is asked for - these words and may paraphrase them. - """ - if self.line is not None and self.id: - await self.line.update(self.id, text, True) - async def disallow_interruptions(self) -> None: """Keep the call going from now on when the caller talks over the agent. diff --git a/sdks/ruby/README.md b/sdks/ruby/README.md index bae53905c0..64980cc09b 100644 --- a/sdks/ruby/README.md +++ b/sdks/ruby/README.md @@ -64,7 +64,6 @@ as its first argument, ahead of the arguments: agent.tools.register("book_flight", description: "Book a flight for the caller", parameters: { type: "object", properties: { flight_id: { type: "string" } } }, timeout: 20, approval: "Book flight {flight_id}?") do |tool_call_context, args| - tool_call_context.update("checking seats") tool_call_context.disallow_interruptions bookings.book(args["flight_id"], user: tool_call_context.user.id, key: tool_call_context.id) end @@ -81,10 +80,7 @@ end `tool_call_context.id` is the id the model gave the call, which a call sent again keeps, so it is the key to make a side effect happen once; `session_id`, `turn_id`, `user` and `conversation` -say where it came from. `update(text)` says how the call is going: a voice agent tells the -caller in its own words once nobody is talking. `say(text)` has it say `text` as written, which -a speech-to-speech model may paraphrase. `disallow_interruptions` keeps the call going from then -on, for a write that must not stop halfway, and raises `CallNotHeldError` when the call had +say where it came from. `disallow_interruptions` keeps the call going from then on, for a write that must not stop halfway, and raises `CallNotHeldError` when the call had already been stopped. A Ruby thread cannot be stopped from outside, so a call that was cancelled or ran out of time keeps running with `tool_call_context.cancelled?` true, and its answer is dropped. A call the router addresses to another watcher (`run: false`) is left alone. @@ -178,30 +174,32 @@ answer it on that session, with the worker's own credential acting for `message. shows it the photos sent with it (`message.images`, by URL). `get_or_create_agent` refuses such a message. -A worker can also run an agent's tools for every session under it, whoever opened it, such -as a conversation started from a browser. The tools are hosted under the agent's config name: +A worker can also run the app's tools, registered once on the client for every agent, for any +session, whoever opened it, such as a conversation started from a browser. The router offers a +tool to a session only where a tool group of its agent's agent.yaml names it: ```ruby -support = GetStream::VisionAgents::Client.new.agent("support") -support.tools.register("lookup_order", description: "An order by id", - parameters: { type: "object", properties: { id: { type: "string" } } }) do |args| +client = GetStream::VisionAgents::Client.new +client.tools.register("lookup_order", description: "An order by id", timeout: 30, + parameters: { type: "object", properties: { id: { type: "string" } } }) do |args| orders.find(args["id"]) end -dispatch = GetStream::VisionAgents::Dispatch.new -dispatch.host(support, tool_timeout: 30) -dispatch.run +client.run_dispatch ``` +A worker that also answers calls or messages hosts them with `dispatch.host(client)` before +`dispatch.run`. + A tool can say who runs it and how it is shown: `executor: "client"` for one a person's device runs (this process still answers it, once the device has reported), and `display_title:` for the words shown while it runs, such as `"Checking your order"`. The tools are declared each time the router says it is ready, and each call runs on its own -thread. `tool_timeout` is how many seconds the router waits for one tool call; nil takes its default of 30 seconds. A hosted tool takes the -same `interruptible:`, `timeout:` and `approval:` (asked on a voice call), is handed its call, -and is cancelled when the router gives up on it. Hosting alone is enough -to `run`, and a router that refuses the tools ends `run` with the reason. +thread. A tool's `timeout:` is how many seconds the router waits for one of its calls; without +one it waits 30 seconds. A hosted tool takes the same `interruptible:` and `approval:` (asked on +a voice call), is handed its call, and is cancelled when the router gives up on it. Hosting +alone is enough to `run`, and a router that refuses the tools ends `run` with the reason. A dispatched call names the session it is for: `join(call)` opens it with voice, on the call the caller is already in. diff --git a/sdks/ruby/lib/getstream/vision_agents/client.rb b/sdks/ruby/lib/getstream/vision_agents/client.rb index d8e660eb62..95947585b8 100644 --- a/sdks/ruby/lib/getstream/vision_agents/client.rb +++ b/sdks/ruby/lib/getstream/vision_agents/client.rb @@ -27,14 +27,18 @@ class Client # Creating a session returns once the agent is in the call, which can take seconds. READ_TIMEOUT = 60 - attr_reader :backend + # The app's tools, registered once for every agent and run by #run_dispatch. An agent is + # offered one only where a tool group of its agent.yaml names it. + attr_reader :backend, :tools # @param backend [Backend] one already built, instead of building one from the options. + # @param tools [Tools] the app's tools, shared with a client this one is derived from. # @param options the Backend options: url, customer_id, api_key, api_secret, token, # user_id and authenticate, each falling back to the environment, and the team and # usage_categories the work is counted against. - def initialize(backend: nil, **options) + def initialize(backend: nil, tools: nil, **options) @backend = backend || Backend.new(**options) + @tools = tools || Tools.new end def get(template, path: {}, query: {}) @@ -81,7 +85,7 @@ def server_side? # credential and one per user, and switching the user on a shared client would make # which user a request was for depend on when it happened to run. def as_user(user, token) - Client.new(backend: @backend.as_user(user, token)) + Client.new(backend: @backend.as_user(user, token), tools: @tools) end # A client acting for a guest, which is what their conversations belong to. @@ -125,6 +129,20 @@ def agent(name, **options) Agent.new(config: name, client: self, **options) end + # Runs the app's #tools for every session they are offered to, whoever opened it, until + # the socket closes. A tool's own timeout is how long the router waits for one of its + # calls; without one it waits 30 seconds. A worker that also answers calls or messages + # hosts the tools with Dispatch#host instead. + # + # client = GetStream::VisionAgents::Client.new + # client.tools.register("lookup_order", description: "An order by id") { |args| orders.find(args["id"]) } + # client.run_dispatch + # + # @raise [Error] when the router refuses the tools. + def run_dispatch + Dispatch.new(client: self).host(self).run + end + # Routes through a stored router config, named by what it was stored under or by its # id. The config holds the targets; an empty name is a router told what to do per call. # No request is made. diff --git a/sdks/ruby/lib/getstream/vision_agents/dispatch.rb b/sdks/ruby/lib/getstream/vision_agents/dispatch.rb index 4de3dfd922..e80c597890 100644 --- a/sdks/ruby/lib/getstream/vision_agents/dispatch.rb +++ b/sdks/ruby/lib/getstream/vision_agents/dispatch.rb @@ -14,7 +14,7 @@ module VisionAgents # dispatch.get_or_create_agent(message) { GetStream::VisionAgents::Agent.new(config: "support") } # .reply(message) # end - # dispatch.host(support) # an Agent whose tools are registered + # dispatch.host(client) # a Client whose tools are registered # dispatch.run # # Each call, message and hosted tool call runs on its own thread. When a call or message @@ -50,20 +50,19 @@ def initialize(capacity: CAPACITY, report_every: REPORT_EVERY, client: nil) @latency = nil end - # Runs an agent's tools for every session opened under it, whoever opened it. + # Runs the app's tools, client.tools, for every session they are offered to, whoever + # opened it. # - # A session's own tools run in the process that opened it. Hosting is the other - # direction: the router offers agent.tools to each session naming the agent's config - # (its name when it has none, as #sync stores it) and sends every call to this worker. - # tool_timeout is how many seconds the router waits for one tool call to be answered - # before telling the model it failed, not how long the worker runs; nil takes the - # router's default of 30 seconds. Call before #run. - def host(agent, tool_timeout: nil) - agent_id = (agent.config || agent.name).to_s - raise ConfigurationError, "hosting needs an agent with a name" if agent_id.empty? - raise ConfigurationError, "#{agent_id} needs at least one tool to host" if agent.tools.empty? - - @lock.synchronize { @hosted << { agent_id: agent_id, tools: agent.tools, tool_timeout: tool_timeout } } + # The tools are registered once on the client, for every agent: the router offers one to + # a session only where a tool group of its agent's agent.yaml names it, and sends every + # call to this worker. Client#run_dispatch does this alone, for a worker with no calls or + # messages to answer. tool_timeout is how many seconds the router waits for one tool call + # to be answered before telling the model it failed, not how long the worker runs; nil + # takes the router's default of 30 seconds. Call before #run. + def host(client, tool_timeout: nil) + raise ConfigurationError, "hosting needs at least one tool on the client" if client.tools.empty? + + @lock.synchronize { @hosted << { tools: client.tools, tool_timeout: tool_timeout } } self end @@ -167,7 +166,7 @@ def received(frame) when "hosting_refused" # A worker whose tools were refused is one nobody will call; saying so beats sitting # connected looking healthy. - raise Error, "the router refused to host tools for agent #{frame['agent_id']}: #{frame['reason']}" + raise Error, "the router refused to host tools: #{frame['reason']}" when "pong" then pong(frame) end end @@ -175,7 +174,7 @@ def received(frame) def offer_hosted @lock.synchronize { @hosted.dup }.each do |offer| timeout_ms = offer[:tool_timeout] ? (offer[:tool_timeout] * 1000).round : 0 - @socket.send_frame(type: "host_tools", agent_id: offer[:agent_id], + @socket.send_frame(type: "host_tools", every_agent: true, tools: offer[:tools].declarations, timeout_ms: timeout_ms) end end diff --git a/sdks/ruby/lib/getstream/vision_agents/generated/spec.rb b/sdks/ruby/lib/getstream/vision_agents/generated/spec.rb index 17a794400c..cb30e82b7f 100644 --- a/sdks/ruby/lib/getstream/vision_agents/generated/spec.rb +++ b/sdks/ruby/lib/getstream/vision_agents/generated/spec.rb @@ -508,7 +508,7 @@ module Generated }.freeze, "getAgentStats" => { method: :get, path: "/v1/stats/agents", - path_params: [].freeze, query: %w[from to].freeze, + path_params: [].freeze, query: %w[from to agent].freeze, body: nil, body_required: false, socket: false, client_accessible: false }.freeze, @@ -1898,7 +1898,7 @@ module Generated required: %w[name].freeze, open: false }.freeze, "AgentConnectorBinding" => { - properties: %w[connection connector_id events name policy require_approval required sandbox timeout_ms tool_patterns tools].freeze, + properties: %w[connection connector_id events loading name output_retention policy require_approval required sandbox timeout_ms tool_patterns tools].freeze, required: %w[name connector_id connection tools].freeze, open: false }.freeze, "AgentConnectorSelection" => { @@ -1942,12 +1942,8 @@ module Generated required: [].freeze, open: false }.freeze, "AgentStats" => { - properties: %w[peak_at peak_concurrent_sessions tool_calls tools].freeze, - required: %w[peak_concurrent_sessions peak_at tool_calls tools].freeze, open: false - }.freeze, - "AgentTools" => { - properties: %w[output_retention progressive].freeze, - required: [].freeze, open: false + properties: %w[avg_turns peak_at peak_concurrent_sessions sessions tool_calls tools].freeze, + required: %w[peak_concurrent_sessions peak_at sessions avg_turns tool_calls tools].freeze, open: false }.freeze, "AgentUser" => { properties: %w[include_phone_history phone phone_history_since user_id].freeze, @@ -2706,7 +2702,7 @@ module Generated required: %w[cost_micros id markdown status].freeze, open: false }.freeze, "OfferedTool" => { - properties: %w[description name parameters source tokens].freeze, + properties: %w[description loading name parameters source tokens].freeze, required: %w[name description tokens].freeze, open: false }.freeze, "OfferedTools" => { @@ -3225,13 +3221,21 @@ module Generated properties: %w[allowed request_id summary tool_call_id turn_id type].freeze, required: %w[allowed request_id tool_call_id type].freeze, open: false }.freeze, + "ToolCallState" => { + properties: %w[approved_at duration_ms error progress rejected_at started_at status tool tool_call_id turn_id type].freeze, + required: %w[type tool_call_id tool turn_id status started_at duration_ms].freeze, open: false + }.freeze, + "ToolGroup" => { + properties: %w[loading name output_retention tools].freeze, + required: %w[name tools].freeze, open: false + }.freeze, "ToolResultCommand" => { properties: %w[error output request_id tool_call_id turn_id type].freeze, required: %w[tool_call_id type].freeze, open: false }.freeze, "ToolStats" => { - properties: %w[avg_ms avg_tokens calls failed p99_ms tool].freeze, - required: %w[tool calls failed avg_tokens avg_ms p99_ms].freeze, open: false + properties: %w[avg_ms avg_tokens avg_turns_before_use calls definition_tokens failed loading p99_ms session_share sessions summary_tokens tool].freeze, + required: %w[tool calls failed sessions session_share avg_turns_before_use avg_tokens avg_ms p99_ms loading definition_tokens summary_tokens].freeze, open: false }.freeze, "TranscriptEntity" => { properties: %w[end_ms start_ms text type].freeze, diff --git a/sdks/ruby/lib/getstream/vision_agents/tool_call.rb b/sdks/ruby/lib/getstream/vision_agents/tool_call.rb index 79f9146c13..36225c3209 100644 --- a/sdks/ruby/lib/getstream/vision_agents/tool_call.rb +++ b/sdks/ruby/lib/getstream/vision_agents/tool_call.rb @@ -14,7 +14,6 @@ class CallNotHeldError < Error; end # The call a tool's block is running for, handed to it as the first argument: # # tools.register("book_flight", description: "Book a flight", ...) do |tool_call_context, args| - # tool_call_context.update("checking seats") # tool_call_context.disallow_interruptions # bookings.book(args["flight_id"], key: tool_call_context.id) unless tool_call_context.cancelled? # end @@ -66,19 +65,6 @@ def cancelled? @lock.synchronize { @state == :cancelled } end - # Says how the call is going. A voice agent tells the caller in its own words once - # nobody is talking, and a conversation shows it on the call's step until the call - # ends. A newer update replaces one not yet said, and the result replaces both. - def update(text) - tell(text, false) - end - - # Has the agent say text as written while the call runs, or post it in a text - # conversation. A speech-to-speech model is asked for these words and may paraphrase. - def say(text) - tell(text, true) - end - # Keeps the call going from now on when the caller talks over the agent, which is what # a write that must not be left halfway wants. Only the call's timeout and its socket # ending stop it after this. @@ -127,15 +113,6 @@ def stop! @answered.broadcast end end - - private - - def tell(text, verbatim) - return nil unless @send && !id.empty? - - @send.call({ type: "tool_update", tool_call_id: id, text: text, verbatim: verbatim }) - nil - end end end end diff --git a/sdks/ruby/test/agent_test.rb b/sdks/ruby/test/agent_test.rb index c641800200..173e85ed55 100644 --- a/sdks/ruby/test/agent_test.rb +++ b/sdks/ruby/test/agent_test.rb @@ -194,7 +194,7 @@ def test_a_cancelled_tool_answers_nothing end end - def test_a_tool_is_handed_its_call_and_says_how_it_is_going + def test_a_tool_is_handed_its_call seen = Thread::Queue.new frames = Thread::Queue.new serve_session do |peer| @@ -202,21 +202,15 @@ def test_a_tool_is_handed_its_call_and_says_how_it_is_going peer.send_frame(type: "tool_call", id: "t1", turn_id: "turn_2", name: "search", arguments: "{}", user: { id: "sam", name: "Sam", custom: { tier: "gold" } }, conversation: { cid: "agent:kept", custom: { line: "support" } }, approved: true) - 3.times { frames << peer.receive(timeout: 5) } + frames << peer.receive(timeout: 5) peer.receive_type("close", timeout: 10) end tools = VA::Tools.new.register("search", description: "Search flights") do |tool_call_context, _args| seen << tool_call_context - tool_call_context.update("checking three airlines") - tool_call_context.say("Still looking, one moment.") "found two" end agent(tools: tools).join(participant_wait_timeout: 0, wait_for_end: false) do - assert_equal({ "type" => "tool_update", "tool_call_id" => "t1", "text" => "checking three airlines", - "verbatim" => false }, frames.pop(timeout: 5)) - assert_equal({ "type" => "tool_update", "tool_call_id" => "t1", "text" => "Still looking, one moment.", - "verbatim" => true }, frames.pop(timeout: 5)) assert_equal "t1", frames.pop(timeout: 5)["tool_call_id"], "the call addressed elsewhere is left" end tool_call_context = seen.pop(timeout: 1) diff --git a/sdks/ruby/test/client_test.rb b/sdks/ruby/test/client_test.rb index 70dc587971..6f46ac4e29 100644 --- a/sdks/ruby/test/client_test.rb +++ b/sdks/ruby/test/client_test.rb @@ -13,6 +13,12 @@ def test_a_request_carries_the_customer_and_the_spec_path assert_equal({ "name" => "docs" }, request.query) end + def test_a_client_acting_for_a_user_shares_the_apps_tools + client.tools.register("weather_lookup", description: "Weather for a place") { "sunny" } + + assert_equal ["weather_lookup"], client.as_user("ada", "token").tools.declarations.map { |tool| tool[:name] } + end + def test_path_parameters_are_escaped @router.on(:get, %r{\A/v1/agents/sessions/}, body: { "id" => "a b/c" }) diff --git a/sdks/ruby/test/dispatch_test.rb b/sdks/ruby/test/dispatch_test.rb index d33f57c74c..00d774cba3 100644 --- a/sdks/ruby/test/dispatch_test.rb +++ b/sdks/ruby/test/dispatch_test.rb @@ -55,9 +55,8 @@ def test_connecting_says_what_the_worker_holds_and_handles end def test_a_worker_that_only_hosts_tools_handles_nothing_and_finishes_work_it_is_handed - agent = client.agent("my-agent") - agent.tools.register("weather_lookup", description: "Weather") { "sunny" } - dispatch.host(agent) + client.tools.register("weather_lookup", description: "Weather") { "sunny" } + dispatch.host(client) run_until_closed do |peer| assert_equal({ "capacity" => "2", "active" => "0", "handles" => "" }, peer.request.query) @@ -236,20 +235,20 @@ def test_a_message_is_answered_by_one_agent_per_channel def test_a_hosted_tool_is_declared_on_every_ready_and_answered_without_blocking_the_socket release = Thread::Queue.new - agent = client.agent("my-agent", name: "Max") - tools = agent.tools + tools = client.tools tools.register("weather_lookup", description: "Weather for a place", parameters: { type: "object", properties: { location: { type: "string" } } }) do |args| { "location" => args["location"], "sky" => "sunny" } end tools.register("slow", description: "Waits to be released") { release.pop(timeout: 5) } tools.register("broken", description: "Always fails") { raise ArgumentError, "no sky today" } - dispatch.host(agent, tool_timeout: 2.5) + dispatch.host(client, tool_timeout: 2.5) run_until_closed do |peer| peer.send_frame(type: "ready", worker_id: "w1") declared = peer.receive_type("host_tools") - assert_equal "my-agent", declared["agent_id"] + assert_equal true, declared["every_agent"] + refute declared.key?("agent_id") assert_equal 2500, declared["timeout_ms"] assert_equal %w[weather_lookup slow broken], declared["tools"].map { |tool| tool["name"] } assert_equal({ "type" => "object", "properties" => { "location" => { "type" => "string" } } }, @@ -270,24 +269,22 @@ def test_a_hosted_tool_is_declared_on_every_ready_and_answered_without_blocking_ assert_equal({ "type" => "tool_result", "tool_call_id" => "t4", "id" => "t4", "error" => "this worker does not run missing" }, peer.receive_type("tool_result")) - peer.send_frame(type: "hosting", agent_id: "my-agent", tools: %w[weather_lookup slow broken]) + peer.send_frame(type: "hosting", every_agent: true, tools: %w[weather_lookup slow broken]) peer.send_frame(type: "ready", worker_id: "w2") - assert_equal "my-agent", peer.receive_type("host_tools")["agent_id"] + assert_equal true, peer.receive_type("host_tools")["every_agent"] end assert_equal "w2", dispatch.worker_id end - def test_a_hosted_tools_policies_call_and_updates_go_over_the_dispatch_socket + def test_a_hosted_tools_policies_and_call_go_over_the_dispatch_socket seen = Thread::Queue.new - agent = client.agent("travel") - agent.tools.register("book_flight", description: "Book a flight", + client.tools.register("book_flight", description: "Book a flight", parameters: { type: "object", properties: { flight_id: { type: "string" } } }, interruptible: true, timeout: 90, approval: "Book flight {flight_id}?") do |tool_call_context, _args| seen << tool_call_context - tool_call_context.update("checking seats") "booked" end - dispatch.host(agent) + dispatch.host(client) run_until_closed do |peer| peer.send_frame(type: "ready", worker_id: "w1") @@ -297,8 +294,6 @@ def test_a_hosted_tools_policies_call_and_updates_go_over_the_dispatch_socket peer.send_frame(type: "tool_call", id: "t1", session_id: "s1", turn_id: "turn_3", name: "book_flight", arguments: "{}", user: { id: "sam" }) - assert_equal({ "type" => "tool_update", "tool_call_id" => "t1", "text" => "checking seats", "verbatim" => false }, - peer.receive_type("tool_update")) assert_equal "booked", peer.receive_type("tool_result")["output"] end tool_call_context = seen.pop(timeout: 1) @@ -308,14 +303,13 @@ def test_a_hosted_tools_policies_call_and_updates_go_over_the_dispatch_socket def test_a_hosted_call_the_router_cancels_is_not_answered release = Thread::Queue.new stopped = Thread::Queue.new - agent = client.agent("support") - agent.tools.register("investigate", description: "Investigate for a while") do |tool_call_context, _args| + client.tools.register("investigate", description: "Investigate for a while") do |tool_call_context, _args| release.pop(timeout: 5) stopped << tool_call_context.cancelled? "found it" end - agent.tools.register("ping", description: "Answer at once") { "pong" } - dispatch.host(agent) + client.tools.register("ping", description: "Answer at once") { "pong" } + dispatch.host(client) run_until_closed do |peer| peer.send_frame(type: "tool_call", id: "t1", session_id: "s1", name: "investigate", arguments: "{}") @@ -330,22 +324,36 @@ def test_a_hosted_call_the_router_cancels_is_not_answered end def test_a_worker_whose_tools_are_refused_stops_with_the_reason - agent = client.agent("my-agent") - agent.tools.register("weather_lookup", description: "Weather for a place") { "sunny" } - dispatch.host(agent) + client.tools.register("weather_lookup", description: "Weather for a place") { "sunny" } + dispatch.host(client) runner = Thread.new { dispatch.run } runner.report_on_exception = false peer = @router.peer peer.send_frame(type: "ready", worker_id: "w1") assert_equal 0, peer.receive_type("host_tools")["timeout_ms"] - peer.send_frame(type: "hosting_refused", agent_id: "my-agent", reason: "another customer's agent") + peer.send_frame(type: "hosting_refused", reason: "hosting no tools is not hosting") refused = assert_raises(VA::Error) { runner.join(5) } - assert_equal "the router refused to host tools for agent my-agent: another customer's agent", refused.message + assert_equal "the router refused to host tools: hosting no tools is not hosting", refused.message + end + + def test_run_dispatch_runs_the_clients_tools_for_every_agent + client.tools.register("weather_lookup", description: "Weather for a place") { "sunny" } + runner = Thread.new { client.run_dispatch } + peer = @router.peer + assert_equal "", peer.request.query["handles"] + peer.send_frame(type: "ready", worker_id: "w1") + declared = peer.receive_type("host_tools") + assert_equal [true, %w[weather_lookup]], [declared["every_agent"], declared["tools"].map { |tool| tool["name"] }] + peer.send_frame(type: "tool_call", id: "t1", session_id: "s1", name: "weather_lookup", arguments: "{}") + assert_equal "sunny", peer.receive_type("tool_result")["output"] + ensure + peer&.close + runner&.join(5) end def test_a_handler_or_hosted_tools_are_needed_before_running assert_raises(VA::ConfigurationError) { dispatch.run } - assert_raises(VA::ConfigurationError) { dispatch.host(client.agent("my-agent")) } + assert_raises(VA::ConfigurationError) { dispatch.host(client) } end end diff --git a/sdks/ruby/test/tool_call_test.rb b/sdks/ruby/test/tool_call_test.rb index 1ed28c4cd0..e552840860 100644 --- a/sdks/ruby/test/tool_call_test.rb +++ b/sdks/ruby/test/tool_call_test.rb @@ -59,22 +59,17 @@ def test_a_block_of_two_is_handed_its_call_first_and_a_block_of_one_only_the_arg def test_a_call_without_a_socket_does_nothing tool_call_context = VA::ToolCallContext.new - assert_nil tool_call_context.update("checking") - assert_nil tool_call_context.say("One moment.") assert_nil tool_call_context.disallow_interruptions refute tool_call_context.cancelled? end - def test_a_call_reports_how_it_is_going_and_a_call_stopped_first_is_not_held + def test_a_call_stopped_first_is_not_held sent = [] tool_call_context = VA::ToolCallContext.new(id: "t1", send: ->(frame) { sent << frame }) - tool_call_context.update("checking") - tool_call_context.say("One moment.") tool_call_context.cancel! - assert_equal [{ type: "tool_update", tool_call_id: "t1", text: "checking", verbatim: false }, - { type: "tool_update", tool_call_id: "t1", text: "One moment.", verbatim: true }], sent + assert_empty sent assert_raises(VA::CallNotHeldError) { tool_call_context.disallow_interruptions } end end diff --git a/sdks/rust/README.md b/sdks/rust/README.md index dafa501a8f..9ae5f70617 100644 --- a/sdks/rust/README.md +++ b/sdks/rust/README.md @@ -76,7 +76,6 @@ tools.register_with_tool_call_context( ..Default::default() }, async |tool_call_context: ToolCallContext, arguments| { - tool_call_context.update("checking seats").await?; tool_call_context.disallow_interruptions().await?; let user = &tool_call_context.user.id; Ok::<_, Error>(book(&arguments["flight_id"], user, &tool_call_context.id).await) @@ -96,10 +95,7 @@ tools.register_with_tool_call_context( `tool_call_context.id` is the id the model gave the call, which a call sent again keeps, so it is the key to make a side effect happen once; `session_id`, `turn_id`, `user` and `conversation` -say where it came from. `update(text)` says how the call is going: a voice agent tells the -caller in its own words once nobody is talking. `say(text)` has it say `text` as written, which -a speech-to-speech model may paraphrase. `disallow_interruptions` keeps the call going from then -on, for a write that must not stop halfway, and returns `Error::NotHeld` when the router had +say where it came from. `disallow_interruptions` keeps the call going from then on, for a write that must not stop halfway, and returns `Error::NotHeld` when the router had already cancelled it. A cancelled call is aborted at its next `.await` and is not answered. A call the router addresses to another watcher (`run: false`) is left alone. @@ -231,26 +227,26 @@ in `images`. `answer` has the model answer it on that session, shown those photo worker's own credential acting for the user who wrote it. `get_or_create_agent` refuses such a message. -A worker can also host an agent's functions for every session opened under its name, -including ones opened from a browser. The router offers them to each session and sends every -call here: +A worker can also host the app's functions, registered once on the client for every agent, for +any session, including ones opened from a browser. The router offers a function to a session +only where a tool group of its agent's agent.yaml names it, and sends every call here: ```rust -let agent = client.agent("my-agent"); -agent.tools.register( +client.tools().register( "get_weather", "The weather in a city", json!({"type": "object", "properties": {"city": {"type": "string"}}}), async |arguments| Ok::<_, String>(format!("sunny in {}", arguments["city"])), ); -dispatch.host(&agent, None); // None: the router waits its default 30 seconds for each tool call -dispatch.run().await?; +client.run_dispatch().await?; ``` A hosted tool registered with `register_with_tool_call_context` takes the same `interruptible`, `timeout_ms` and `approval` (asked on a voice call), is handed its call, and is cancelled when -the router gives up on it. A worker that only hosts tools needs no handler. If the router -refuses the tools, `run` returns `Error::Failed` naming the agent and the reason. +the router gives up on it. Without its own `timeout_ms` the router waits 30 seconds for a call. +A worker that also answers calls or messages hosts the tools with `dispatch.host(&client, None)` +before `run`. If the router refuses the tools, `run_dispatch` returns `Error::Failed` with the +reason. Every caller to an attached number gets a session of their own: the `call` handed to a worker carries its `session_id`, and `answer` joins that session with voice on. `wait_for_call(number)` diff --git a/sdks/rust/src/agent.rs b/sdks/rust/src/agent.rs index be51a61a3f..53098d4b2c 100644 --- a/sdks/rust/src/agent.rs +++ b/sdks/rust/src/agent.rs @@ -775,6 +775,8 @@ async fn connector_bindings( policy: None, require_approval: None, sandbox: None, + loading: None, + output_retention: None, timeout_ms: None, }); } diff --git a/sdks/rust/src/client.rs b/sdks/rust/src/client.rs index 5b4803bfcb..f5e588e177 100644 --- a/sdks/rust/src/client.rs +++ b/sdks/rust/src/client.rs @@ -8,6 +8,7 @@ use serde::de::DeserializeOwned; use serde_json::json; use crate::backend::{Backend, ClientOptions, checked_categories, sign_for, with_usage}; +use crate::dispatch::Dispatch; use crate::error::{Error, Result}; use crate::limits::SpendLimits; use crate::router::Router; @@ -15,6 +16,7 @@ use crate::sessions::{AgentRef, Memories}; use crate::simulations::Simulations; use crate::socket::Socket; use crate::teams::Teams; +use crate::tools::Tools; use crate::types; /// The router, held once: where it is and who is calling it. @@ -31,10 +33,15 @@ pub struct Client { struct Inner { backend: Backend, http: reqwest::Client, + tools: Tools, } impl Client { pub fn new(options: ClientOptions) -> Result { + Client::with_tools(options, Tools::new()) + } + + fn with_tools(options: ClientOptions, tools: Tools) -> Result { let backend = Backend::new(&options)?; let http = match options.http { Some(http) => http, @@ -48,7 +55,11 @@ impl Client { })?, }; Ok(Client { - inner: Arc::new(Inner { backend, http }), + inner: Arc::new(Inner { + backend, + http, + tools, + }), }) } @@ -85,20 +96,23 @@ impl Client { ))); } let backend = &self.inner.backend; - Client::new(ClientOptions { - url: Some(backend.url.clone()), - region: None, - customer_id: Some(backend.customer_id.clone()), - api_key: Some(backend.api_key.clone()), - // A token is the whole credential, so holding one stops this being a backend. - api_secret: Some(String::new()), - token: Some(token.to_string()), - user_id: Some(user_id.to_string()), - authenticate: Some(backend.authenticate), - team: None, - usage_categories: Vec::new(), - http: Some(self.inner.http.clone()), - }) + Client::with_tools( + ClientOptions { + url: Some(backend.url.clone()), + region: None, + customer_id: Some(backend.customer_id.clone()), + api_key: Some(backend.api_key.clone()), + // A token is the whole credential, so holding one stops this being a backend. + api_secret: Some(String::new()), + token: Some(token.to_string()), + user_id: Some(user_id.to_string()), + authenticate: Some(backend.authenticate), + team: None, + usage_categories: Vec::new(), + http: Some(self.inner.http.clone()), + }, + self.inner.tools.clone(), + ) } /// A token for one end user to hold, naming their team and usage categories, valid for @@ -157,6 +171,7 @@ impl Client { inner: Arc::new(Inner { backend, http: self.inner.http.clone(), + tools: self.inner.tools.clone(), }), } } @@ -174,6 +189,35 @@ impl Client { AgentRef::new(self.clone(), name) } + /// The app's tools, registered once for every agent and run by + /// [`run_dispatch`](Client::run_dispatch). An agent is offered one only where a tool group + /// of its agent.yaml names it. + pub fn tools(&self) -> &Tools { + &self.inner.tools + } + + /// Runs the app's [`tools`](Client::tools) for every session they are offered to, whoever + /// opened it, until the router closes the connection. A tool's own timeout is how long the + /// router waits for one of its calls; without one it waits 30 seconds. A worker that also + /// answers calls or messages hosts the tools with [`Dispatch::host`] instead. + /// + /// ```no_run + /// # async fn example(client: vision_agents::Client) -> vision_agents::Result<()> { + /// client.tools().register( + /// "get_weather", + /// "The weather in a city", + /// serde_json::json!({"type": "object", "properties": {"city": {"type": "string"}}}), + /// async |arguments| Ok::<_, String>(format!("sunny in {}", arguments["city"])), + /// ); + /// client.run_dispatch().await + /// # } + /// ``` + pub async fn run_dispatch(&self) -> Result<()> { + let dispatch = Dispatch::new(self.clone()); + dispatch.host(self, None); + dispatch.run().await + } + /// The router, routing through the stored router config named `config` (by name or id), /// which holds the target. An empty name is a router told what to do per call instead. pub fn router(&self, config: &str) -> Router { diff --git a/sdks/rust/src/dispatch.rs b/sdks/rust/src/dispatch.rs index 2bfd35b10f..51c396b59c 100644 --- a/sdks/rust/src/dispatch.rs +++ b/sdks/rust/src/dispatch.rs @@ -17,7 +17,6 @@ use crate::client::Client; use crate::error::{Error, Result}; use crate::responses::{AgentResponse, Responses}; use crate::session::Session; -use crate::sessions::AgentRef; use crate::socket::{Frame, Incoming, SocketSender}; use crate::tool_call::{Holds, Line, ToolCallContext, to_run}; use crate::tools::Tools; @@ -152,10 +151,9 @@ fn images_of(frame: &Frame) -> Vec { type CallHandler = Arc BoxFuture<'static, Result<()>> + Send + Sync>; type MessageHandler = Arc BoxFuture<'static, Result<()>> + Send + Sync>; -/// One set of functions this worker runs for every session under an agent id. +/// One set of functions this worker runs for every agent of the app. #[derive(Clone)] struct Hosting { - agent_id: String, tools: Tools, tool_timeout: Option, } @@ -184,7 +182,7 @@ impl Hosting { let timeout_ms = self .tool_timeout .map_or(0, |timeout| timeout.as_millis() as u64); - json!({"type": "host_tools", "agent_id": self.agent_id, "tools": tools, "timeout_ms": timeout_ms}) + json!({"type": "host_tools", "every_agent": true, "tools": tools, "timeout_ms": timeout_ms}) } fn runs(&self, name: &str) -> bool { @@ -291,36 +289,35 @@ impl Dispatch { self } - /// Runs the agent's [`tools`](AgentRef::tools) for every session opened under its name, - /// whoever opened it. + /// Runs the app's [`tools`](Client::tools) for every session they are offered to, whoever + /// opened it. /// - /// A session's own functions run in the process that opened it, which is no use to a - /// conversation opened from a browser. Hosting is the other direction: the router offers - /// these functions to each session naming the agent and sends every call to a worker - /// hosting them. `tool_timeout` is how long the router waits for one tool call to be - /// answered before telling the model it failed, not how long the worker runs; `None` - /// takes the router's default of 30 seconds. Call before [`Dispatch::run`]. + /// The tools are registered once on the client, for every agent: the router offers one to + /// a session only where a tool group of its agent's agent.yaml names it, and sends every + /// call to a worker hosting it. [`Client::run_dispatch`] does this alone, for a worker + /// with no calls or messages to answer. `tool_timeout` is how long the router waits for + /// one tool call to be answered before telling the model it failed, not how long the + /// worker runs; `None` takes the router's default of 30 seconds. Call before + /// [`Dispatch::run`]. /// /// ```no_run /// # async fn example(client: vision_agents::Client) -> vision_agents::Result<()> { /// use vision_agents::Dispatch; /// - /// let agent = client.agent("my-agent"); - /// agent.tools.register( + /// client.tools().register( /// "get_weather", /// "The weather in a city", /// serde_json::json!({"type": "object", "properties": {"city": {"type": "string"}}}), /// async |arguments| Ok::<_, String>(format!("sunny in {}", arguments["city"])), /// ); - /// let dispatch = Dispatch::new(client); - /// dispatch.host(&agent, None); + /// let dispatch = Dispatch::new(client.clone()); + /// dispatch.host(&client, None); /// dispatch.run().await /// # } /// ``` - pub fn host(&self, agent: &AgentRef, tool_timeout: Option) -> &Self { + pub fn host(&self, client: &Client, tool_timeout: Option) -> &Self { self.inner.hosted.lock().expect("hosted").push(Hosting { - agent_id: agent.name.clone(), - tools: agent.tools.clone(), + tools: client.tools().clone(), tool_timeout, }); self @@ -558,8 +555,7 @@ impl Dispatch { failure = Some(Error::Failed { operation: "dispatch".into(), message: format!( - "the router refused to host tools for agent {}: {}", - frame.text("agent_id"), + "the router refused to host tools: {}", frame.text("reason") ), }); diff --git a/sdks/rust/src/operations.rs b/sdks/rust/src/operations.rs index 5908158e90..94b7da0ab6 100644 --- a/sdks/rust/src/operations.rs +++ b/sdks/rust/src/operations.rs @@ -5145,6 +5145,9 @@ pub struct GetAgentStatsQuery { /// End of the window, exclusive. #[serde(skip_serializing_if = "Option::is_none")] pub to: Option, + /// The id of an agent config: only its sessions are counted. Omitted is every agent. + #[serde(skip_serializing_if = "Option::is_none")] + pub agent: Option, } /// The query string `getSpend` takes. A field left `None` is left out. #[derive(Debug, Clone, Default, PartialEq, Serialize)] diff --git a/sdks/rust/src/sessions.rs b/sdks/rust/src/sessions.rs index 1988429bf7..3b2c43a597 100644 --- a/sdks/rust/src/sessions.rs +++ b/sdks/rust/src/sessions.rs @@ -21,9 +21,6 @@ pub struct AgentRef { pub name: String, /// This agent's conversations: opening one, and reading the old ones back. pub sessions: Sessions, - /// Functions run for every session of this agent, whoever opened it, once a dispatch - /// worker hosts them with [`Dispatch::host`](crate::Dispatch::host). - pub tools: Tools, client: Client, } @@ -35,7 +32,6 @@ impl AgentRef { client: client.clone(), agent: name.into(), }, - tools: Tools::new(), client, } } diff --git a/sdks/rust/src/tool_call.rs b/sdks/rust/src/tool_call.rs index 1f1d6e3596..e1e12fadd4 100644 --- a/sdks/rust/src/tool_call.rs +++ b/sdks/rust/src/tool_call.rs @@ -66,17 +66,6 @@ impl ToolCallContext { } } - /// Tells the people in the conversation how the call is going, such as "Found three - /// flights, comparing prices". The agent puts it in its own words. - pub async fn update(&self, text: &str) -> Result<()> { - self.send_update(text, false).await - } - - /// [`ToolCallContext::update`], said word for word. - pub async fn say(&self, text: &str) -> Result<()> { - self.send_update(text, true).await - } - /// Keeps the call going if the person talks over the agent from here on, for a call /// that is past the point of taking back, such as one that has started a payment. /// @@ -98,15 +87,6 @@ impl ToolCallContext { _ => Err(Error::NotHeld(self.id.clone())), } } - - async fn send_update(&self, text: &str, verbatim: bool) -> Result<()> { - let Some(line) = &self.line else { - return Ok(()); - }; - line.sender - .send(&json!({"type": "tool_update", "tool_call_id": self.id, "text": text, "verbatim": verbatim})) - .await - } } /// A call's way back over the socket that delivered it. diff --git a/sdks/rust/src/types.rs b/sdks/rust/src/types.rs index 0300fe2408..bd0026dcc8 100644 --- a/sdks/rust/src/types.rs +++ b/sdks/rust/src/types.rs @@ -581,8 +581,9 @@ pub struct AgentConfig { skip_serializing_if = ":: std :: collections :: BTreeMap::is_empty" )] pub tags: ::std::collections::BTreeMap<::std::string::String, ::std::string::String>, + ///The functions an SDK provides that the agent may offer, in named groups. Empty when there are none, and then the agent offers no function. #[serde(skip_serializing_if = "::std::option::Option::is_none")] - pub tools: ::std::option::Option, + pub tools: ::std::option::Option<::std::vec::Vec>, #[serde(skip_serializing_if = "::std::option::Option::is_none")] pub tts: ::std::option::Option<::std::string::String>, ///Absent when the agent is taught no UI. @@ -692,8 +693,9 @@ pub struct AgentConfigPatch { skip_serializing_if = ":: std :: collections :: BTreeMap::is_empty" )] pub tags: ::std::collections::BTreeMap<::std::string::String, ::std::string::String>, + ///Replaces the tool groups whole; an empty list removes them all. #[serde(skip_serializing_if = "::std::option::Option::is_none")] - pub tools: ::std::option::Option, + pub tools: ::std::option::Option<::std::vec::Vec>, #[serde(skip_serializing_if = "::std::option::Option::is_none")] pub tts: ::std::option::Option<::std::string::String>, ///What teaches the agent's model to answer with UI. An empty string removes them. @@ -828,9 +830,9 @@ pub struct AgentConfigRequest { skip_serializing_if = ":: std :: collections :: BTreeMap::is_empty" )] pub tags: ::std::collections::BTreeMap<::std::string::String, ::std::string::String>, - ///How the agent is offered its connector tools. Left out on an update, the stored settings stay. + ///The functions an SDK provides that the agent may offer, in named groups, each with how it is loaded. A function an SDK registers, declares on a session or hosts in a worker is offered only when a group names it: a session declaring one that none names is refused (400 tool_not_configured), and a worker's is left out. The router's own tools, connector tools and other agents are not named here. Omitted or null on an update, the groups stored stay; an empty list removes them all. The mapping this field once was, with progressive and output_retention, is refused: loading summary and output_retention on a group or a connector binding replaced them. #[serde(skip_serializing_if = "::std::option::Option::is_none")] - pub tools: ::std::option::Option, + pub tools: ::std::option::Option<::std::vec::Vec>, #[serde(skip_serializing_if = "::std::option::Option::is_none")] pub tts: ::std::option::Option<::std::string::String>, ///What teaches the agent's model to answer with UI, such as the section the SDK compiles from agent.yaml's ui key. Every session is told them after the instructions, unless it was opened with ui false. Empty teaches none. Left out on an update, the stored ones stay. @@ -868,8 +870,14 @@ pub struct AgentConnectorBinding { ///MCP events the binding's fixed connection is subscribed to, each opening a text conversation from the config when it arrives. Subscribed when the connection is next validated. Only a fixed binding may declare events: a session binding's connection is picked when a session opens, and an event arrives with no session open. #[serde(skip_serializing_if = "::std::option::Option::is_none")] pub events: ::std::option::Option<::std::vec::Vec>, + ///How the binding's tools are offered to the model. Omitted is full. + #[serde(skip_serializing_if = "::std::option::Option::is_none")] + pub loading: ::std::option::Option, ///The alias, unique within the config: a lowercase letter, then up to 62 lowercase letters, digits, - or _, never __ and not ending in _. The model is offered each tool as __, split back at the first __, so a __ inside the alias or a _ at its end would split it in the wrong place. pub name: ::std::string::String, + ///How long the agent's sessions keep what the binding's tools were given and gave back on the turns they record, as a tool group's output_retention. Omitted or empty, it is kept as long as the session. + #[serde(skip_serializing_if = "::std::option::Option::is_none")] + pub output_retention: ::std::option::Option<::std::string::String>, ///How the binding's tool calls behave around speech and interruptions. Omitted, a call is cancelled at the provider when the turn is interrupted, and the agent picks its own words while it runs. #[serde(skip_serializing_if = "::std::option::Option::is_none")] pub policy: ::std::option::Option, @@ -1386,26 +1394,21 @@ pub struct AgentScheduling { ///`AgentStats` #[derive(::serde::Deserialize, ::serde::Serialize, Clone, Debug, PartialEq)] pub struct AgentStats { + ///Turns a session took in the window, on average. A turn is a reply to the person, not a rewound one or a delegated task's. Null when no session took one. + #[serde(deserialize_with = "::std::option::Option::deserialize")] + pub avg_turns: ::std::option::Option, ///When that peak was first reached. Null when no session was open in the window. #[serde(deserialize_with = "::std::option::Option::deserialize")] pub peak_at: ::std::option::Option<::std::string::String>, ///The most sessions open at the same time, each one an agent running. A session still open counts up to now. pub peak_concurrent_sessions: i64, + ///Sessions that took a turn in the window: what a tool's session_share is out of. + pub sessions: i64, ///Every tool call in the window, as one. Its tool is empty. pub tool_calls: ToolStats, ///One row per tool, most called first. pub tools: ::std::vec::Vec, } -///How an agent is offered its tools, and how long what they were given and gave back is kept. -#[derive(::serde::Deserialize, ::serde::Serialize, Clone, Debug, Default, PartialEq)] -pub struct AgentTools { - ///How long this agent's sessions keep what any of its tools were given and gave back on the turns they record: a tool's arguments, its result, a search's query and the wording of a tool's error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed. Left out on a create, or empty, it is kept as long as the session; left out on an update, the stored setting stays, and an empty string removes it. The app's and the organization's policies (tool_output_retention) can shorten it and never lengthen it, and the shortest applies. It applies to tool calls recorded after it is set. - #[serde(skip_serializing_if = "::std::option::Option::is_none")] - pub output_retention: ::std::option::Option<::std::string::String>, - ///Offer each tool by the first line of its description, with its arguments' descriptions left out, and have the first call to a tool return its full description and input schema instead of running it. It saves context on an agent with many tools, at the cost of one more model turn for each tool a conversation uses. Off by default. Left out on an update, the stored setting stays. - #[serde(skip_serializing_if = "::std::option::Option::is_none")] - pub progressive: ::std::option::Option, -} ///`AgentUser` #[derive(::serde::Deserialize, ::serde::Serialize, Clone, Debug, PartialEq, Default)] pub struct AgentUser { @@ -5882,7 +5885,7 @@ pub enum EqualsOrIn { ///`ErrorDetail` #[derive(::serde::Deserialize, ::serde::Serialize, Clone, Debug, PartialEq)] pub struct ErrorDetail { - ///What went wrong, for a program to branch on. Every type has a code of its own name (invalid_request, unauthenticated, forbidden, payment_required, not_found, method_not_allowed, not_acceptable, conflict, gone, payload_too_large, unsupported_media_type, rate_limited, internal_error, unavailable, timeout) that a failure has when nothing names it better. The others are validation_failed, missing_customer, missing_organization, server_side_only, not_configured (this deployment does not offer the feature), modality_not_routed, unsynced_changes (a sync asked to check would write over somebody's edits), name_taken (a 409: another agent config, router config or voice, or another skill of the same agent config, already has the name, so another name will do), channel_connection_taken (a 409: another live agent config binds the channel connection as fixed, and one agent answers a channel connection's messages, so the message names that config), call_unanswered (a 409: the call placed for a phone session ended with nobody answering, and the message names how: busy, no_answer, failed or canceled. Opening the session is refused, or, when the status came while the agent was joining the call, the session is ended at once and kept as ended), no_oauth_client (a 400: a consent found no OAuth client for the connector, and the message names each way the connector lets the app give it one), invalid_config (a 400: an agent config a run names does not start a session), insufficient_credits (a 402: the organization's credit is spent, so work that costs money is refused until credit is added; a model that costs nothing is still served), system_limit_reached (a 402: the organization's spend hard limit for the day or the month is reached, whatever its credit, and the message names the interval, the amount and when it resets), spend_limit_reached (a 402: one of the organization's own spend limits is spent, on the organization, an app, a key, a usage category or a team, and the message says which and when it resets, so retrying before then is refused again), user_limit_reached (a 429: a spend limit on the one end user the request is for is spent, which stops that user and no one else, and Retry-After says when its window ends), spend_check_unavailable (a 503: the router could not share what the organization is spending with the other routers for a while, so work that costs money is refused until it can; a model that costs nothing is still served, and Retry-After says to try again in a few seconds), budget_moved (a 400: a policy sent a budget, which spend limits replaced), limit_above_parent (a 400: a spend limit is above a limit of its parent in the same interval, and the message names the parent's limit), limit_exists (a 409: the subject already has a limit at that level and interval, and the message names its id), no_limit (a 404: the subject has no spend limit to reset or grant past), use_team (a 400: a category starting team: was sent, where the team field names a team), invalid_expiry (a 400: a grant's expires_at is not in the future, or is more than 366 days away), too_many_grants (a 409: the organization has 20 live grants, as many as it may), tier_limit_reached (the organization's tier limits refuse the work, and limit says which: a 403 for a call or a transfer, to a country the tier does not reach, past its minutes or recipients of the day, or past its calls at once; a 400 for an agent config whose sandbox is bigger or more open than the organization's sandbox limits allow. The message names the tier and the limit), account_suspended (a 403: the organization is suspended, and every request is refused, one for a model that costs nothing included, until it is restored; the message says to contact support), concurrency_limited (a 429: the app has as many runs in flight as it may, and Retry-After says when to try again), model_failed (a 503: the model failed or gave no answer), prompt_injection (a 400: the prompt injection screen refused the input of a run or a tool result it read, which retrying an unchanged input does not fix), image_unavailable (a 400: an image a run was handed could not be fetched or is not a JPEG, PNG, WebP or GIF, which retrying the same image does not fix), document_too_large (a 413: a file sent with a response has more pages or text than one may, alone or with the files the conversation already holds, and the message names the limit), document_unreadable (a 400: a file sent with a response could not be read, such as one with a password, one that is damaged or one with no text in it, which retrying the same file does not fix), invalid_usage_categories (a 400: the usage categories the request names, in the token's usage_categories and team claims or in X-Stream-Usage-Categories, are more than 10, one is not 1 to 64 characters of A-Z, a-z, 0-9, _ . : -, more than one names a team, or the team is over 59 characters), memory_unavailable (a 503: the memory store could not delete what was asked, so nothing that waits on it was deleted either, and the same request later can succeed), billing_not_enabled (a 404: this deployment bills nobody, or the caller's app is in no organization it bills, so there is no account to read), provider_takes_no_key (a 400: a key was put for a provider whose work is never routed on a customer's own key here, and the message names the ones that are), campaign_locked (a 409: the campaign's state does not allow the change, such as a launched campaign's sequence, or resuming one that is running), campaign_not_ready (a 400: a launch found the campaign missing what it needs to ring anybody, and the message says what), consent_required (a 400: a launch did not say the people on the campaign agreed to be called), branch_limit_reached (a 409: the conversation has as many edit and regenerate points as Stream Chat keeps for it, 100, so a regenerate or an edit is refused and the conversation is continued in a new chat; asking and reading are unaffected), branch_too_long (a 409: the branch an activation would bring back is more than 100 messages to send again, so it is continued in a new chat), branch_unavailable (a 410: the copies of a hidden branch were deleted, as a memory truncate deletes them, so it can be neither read nor brought back), switch_unfinished (a 409: a regenerate, an edit or an activation of the session failed after Stream Chat may have rewound, and finishing it still fails because Stream Chat does not answer, so the session's questions, regenerates, edits and activations are refused meanwhile; try again: each one tries to finish it first, so it clears once Stream Chat answers), and _not_found for agent_config, call, campaign, channel_account, command, connection, grant, knowledge_document, knowledge_url, limit, provider_key, router_config, scheduled_task, session, simulation, simulation_run, skill and voice. A scheduled task also has invalid_schedule (a schedule that does not read: at and in both or neither, a date, time or duration that does not parse, or until without repeat), schedule_in_past, schedule_too_far (a one-shot, or an in, more than a year ahead), invalid_timezone, timezone_unknown (no zone was given and neither the user nor the config has one), invalid_repeat, interval_too_short (a run moved to less than 15 minutes from now and earlier than the run it replaces) and too_many_tasks (a 409: the user is at their active-task limit). More may be added, so a client should expect one it does not know. + ///What went wrong, for a program to branch on. Every type has a code of its own name (invalid_request, unauthenticated, forbidden, payment_required, not_found, method_not_allowed, not_acceptable, conflict, gone, payload_too_large, unsupported_media_type, rate_limited, internal_error, unavailable, timeout) that a failure has when nothing names it better. The others are validation_failed, missing_customer, missing_organization, server_side_only, not_configured (this deployment does not offer the feature), modality_not_routed, unsynced_changes (a sync asked to check would write over somebody's edits), name_taken (a 409: another agent config, router config or voice, or another skill of the same agent config, already has the name, so another name will do), channel_connection_taken (a 409: another live agent config binds the channel connection as fixed, and one agent answers a channel connection's messages, so the message names that config), call_unanswered (a 409: the call placed for a phone session ended with nobody answering, and the message names how: busy, no_answer, failed or canceled. Opening the session is refused, or, when the status came while the agent was joining the call, the session is ended at once and kept as ended), no_oauth_client (a 400: a consent found no OAuth client for the connector, and the message names each way the connector lets the app give it one), invalid_config (a 400: an agent config a run names does not start a session), tool_not_configured (a 400: a session under an agent config was declared or given a function that none of the config's tool groups names, and the message names it), insufficient_credits (a 402: the organization's credit is spent, so work that costs money is refused until credit is added; a model that costs nothing is still served), system_limit_reached (a 402: the organization's spend hard limit for the day or the month is reached, whatever its credit, and the message names the interval, the amount and when it resets), spend_limit_reached (a 402: one of the organization's own spend limits is spent, on the organization, an app, a key, a usage category or a team, and the message says which and when it resets, so retrying before then is refused again), user_limit_reached (a 429: a spend limit on the one end user the request is for is spent, which stops that user and no one else, and Retry-After says when its window ends), spend_check_unavailable (a 503: the router could not share what the organization is spending with the other routers for a while, so work that costs money is refused until it can; a model that costs nothing is still served, and Retry-After says to try again in a few seconds), budget_moved (a 400: a policy sent a budget, which spend limits replaced), limit_above_parent (a 400: a spend limit is above a limit of its parent in the same interval, and the message names the parent's limit), limit_exists (a 409: the subject already has a limit at that level and interval, and the message names its id), no_limit (a 404: the subject has no spend limit to reset or grant past), use_team (a 400: a category starting team: was sent, where the team field names a team), invalid_expiry (a 400: a grant's expires_at is not in the future, or is more than 366 days away), too_many_grants (a 409: the organization has 20 live grants, as many as it may), tier_limit_reached (the organization's tier limits refuse the work, and limit says which: a 403 for a call or a transfer, to a country the tier does not reach, past its minutes or recipients of the day, or past its calls at once; a 400 for an agent config whose sandbox is bigger or more open than the organization's sandbox limits allow. The message names the tier and the limit), account_suspended (a 403: the organization is suspended, and every request is refused, one for a model that costs nothing included, until it is restored; the message says to contact support), concurrency_limited (a 429: the app has as many runs in flight as it may, and Retry-After says when to try again), model_failed (a 503: the model failed or gave no answer), prompt_injection (a 400: the prompt injection screen refused the input of a run or a tool result it read, which retrying an unchanged input does not fix), image_unavailable (a 400: an image a run was handed could not be fetched or is not a JPEG, PNG, WebP or GIF, which retrying the same image does not fix), document_too_large (a 413: a file sent with a response has more pages or text than one may, alone or with the files the conversation already holds, and the message names the limit), document_unreadable (a 400: a file sent with a response could not be read, such as one with a password, one that is damaged or one with no text in it, which retrying the same file does not fix), invalid_usage_categories (a 400: the usage categories the request names, in the token's usage_categories and team claims or in X-Stream-Usage-Categories, are more than 10, one is not 1 to 64 characters of A-Z, a-z, 0-9, _ . : -, more than one names a team, or the team is over 59 characters), memory_unavailable (a 503: the memory store could not delete what was asked, so nothing that waits on it was deleted either, and the same request later can succeed), billing_not_enabled (a 404: this deployment bills nobody, or the caller's app is in no organization it bills, so there is no account to read), provider_takes_no_key (a 400: a key was put for a provider whose work is never routed on a customer's own key here, and the message names the ones that are), campaign_locked (a 409: the campaign's state does not allow the change, such as a launched campaign's sequence, or resuming one that is running), campaign_not_ready (a 400: a launch found the campaign missing what it needs to ring anybody, and the message says what), consent_required (a 400: a launch did not say the people on the campaign agreed to be called), branch_limit_reached (a 409: the conversation has as many edit and regenerate points as Stream Chat keeps for it, 100, so a regenerate or an edit is refused and the conversation is continued in a new chat; asking and reading are unaffected), branch_too_long (a 409: the branch an activation would bring back is more than 100 messages to send again, so it is continued in a new chat), branch_unavailable (a 410: the copies of a hidden branch were deleted, as a memory truncate deletes them, so it can be neither read nor brought back), switch_unfinished (a 409: a regenerate, an edit or an activation of the session failed after Stream Chat may have rewound, and finishing it still fails because Stream Chat does not answer, so the session's questions, regenerates, edits and activations are refused meanwhile; try again: each one tries to finish it first, so it clears once Stream Chat answers), and _not_found for agent_config, call, campaign, channel_account, command, connection, grant, knowledge_document, knowledge_url, limit, provider_key, router_config, scheduled_task, session, simulation, simulation_run, skill and voice. A scheduled task also has invalid_schedule (a schedule that does not read: at and in both or neither, a date, time or duration that does not parse, or until without repeat), schedule_in_past, schedule_too_far (a one-shot, or an in, more than a year ahead), invalid_timezone, timezone_unknown (no zone was given and neither the user nor the config has one), invalid_repeat, interval_too_short (a run moved to less than 15 minutes from now and earlier than the run it replaces) and too_many_tasks (a 409: the user is at their active-task limit). More may be added, so a client should expect one it does not know. pub code: ::std::string::String, ///Where the code is explained. pub doc_url: ::std::string::String, @@ -8423,6 +8426,9 @@ impl ::std::convert::TryFrom<::std::string::String> for OcrStatus { pub struct OfferedTool { ///What the model is told the tool does. pub description: ::std::string::String, + ///How the tool is offered, as its tool group or connector binding says: full for the router's own. A deferred tool is listed once the model has loaded it. Absent for a session that has ended. + #[serde(skip_serializing_if = "::std::option::Option::is_none")] + pub loading: ::std::option::Option, ///How the model asks for it. pub name: ::std::string::String, ///The JSON Schema of its arguments. @@ -8926,7 +8932,7 @@ pub struct Policy { skip_serializing_if = ":: std :: collections :: BTreeMap::is_empty" )] pub tags: ::std::collections::BTreeMap<::std::string::String, ::std::string::String>, - ///How long a session keeps what its tools were given and gave back on the turns it records: a tool's arguments, its result, a search's query and the wording of a tool's error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed, so a session's items read in the same order and tool usage still counts the call. Left out, or empty, it is kept as long as the session. The shortest of the organization's, the app's and the agent config's tools.output_retention applies, so an agent can shorten it and never lengthen it. It applies to tool calls recorded after it is set. Deleting a session deletes all of it at once. + ///How long a session keeps what its tools were given and gave back on the turns it records: a tool's arguments, its result, a search's query and the wording of a tool's error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed, so a session's items read in the same order and tool usage still counts the call. Left out, or empty, it is kept as long as the session. The shortest of the organization's, the app's and the output_retention of the agent config's tool group or connector binding the tool is in applies, so an agent can shorten it and never lengthen it. It applies to tool calls recorded after it is set. Deleting a session deletes all of it at once. #[serde(skip_serializing_if = "::std::option::Option::is_none")] pub tool_output_retention: ::std::option::Option<::std::string::String>, } @@ -13183,9 +13189,9 @@ pub struct SyncAgentRequest { skip_serializing_if = ":: std :: collections :: BTreeMap::is_empty" )] pub tags: ::std::collections::BTreeMap<::std::string::String, ::std::string::String>, - ///How connector tools are offered. A setting left out keeps what is stored. + ///agent.yaml's tools: the functions an SDK provides that the agent may offer, in named groups, as AgentConfigRequest's tools names them, with the same rules. Sent, they replace the stored groups whole, an empty list removing them. Left out, the stored ones stay. #[serde(skip_serializing_if = "::std::option::Option::is_none")] - pub tools: ::std::option::Option, + pub tools: ::std::option::Option<::std::vec::Vec>, #[serde(skip_serializing_if = "::std::option::Option::is_none")] pub tts: ::std::option::Option<::std::string::String>, ///What teaches the agent's model to answer with UI, which the SDK compiles from agent.yaml's ui key. Kept apart from instructions, so a session opened with ui false is not told them. Left out, the agent is taught none. @@ -13540,6 +13546,203 @@ impl ::std::convert::TryFrom<::std::string::String> for ToolApprovalCommandType value.parse() } } +///tool_call.started is the call's first frame, tool_call.progress one while it runs or waits, and tool_call.finished its last. +#[derive( + ::serde::Deserialize, ::serde::Serialize, Clone, Debug, Eq, Hash, Ord, PartialEq, PartialOrd, +)] +pub enum ToolCallFrameType { + #[serde(rename = "tool_call.started")] + ToolCallStarted, + #[serde(rename = "tool_call.progress")] + ToolCallProgress, + #[serde(rename = "tool_call.finished")] + ToolCallFinished, + /// A value this SDK's spec does not list, as the router sent it. + #[serde(untagged)] + Unlisted(::std::string::String), +} +impl ::std::fmt::Display for ToolCallFrameType { + fn fmt(&self, f: &mut ::std::fmt::Formatter<'_>) -> ::std::fmt::Result { + match *self { + Self::ToolCallStarted => f.write_str("tool_call.started"), + Self::ToolCallProgress => f.write_str("tool_call.progress"), + Self::ToolCallFinished => f.write_str("tool_call.finished"), + Self::Unlisted(ref value) => f.write_str(value), + } + } +} +impl ::std::str::FromStr for ToolCallFrameType { + type Err = self::error::ConversionError; + fn from_str(value: &str) -> ::std::result::Result { + match value { + "tool_call.started" => Ok(Self::ToolCallStarted), + "tool_call.progress" => Ok(Self::ToolCallProgress), + "tool_call.finished" => Ok(Self::ToolCallFinished), + _ => Err("invalid value".into()), + } + } +} +impl ::std::convert::TryFrom<&str> for ToolCallFrameType { + type Error = self::error::ConversionError; + fn try_from(value: &str) -> ::std::result::Result { + value.parse() + } +} +impl ::std::convert::TryFrom<::std::string::String> for ToolCallFrameType { + type Error = self::error::ConversionError; + fn try_from( + value: ::std::string::String, + ) -> ::std::result::Result { + value.parse() + } +} +///One change to a tool call's state. Every watcher is sent tool_call.started once, tool_call.progress for each change while it runs or waits, and tool_call.finished once, carrying the call's whole state each time, so a client keeps the last one it got per tool_call_id. +#[derive(::serde::Deserialize, ::serde::Serialize, Clone, Debug, PartialEq)] +pub struct ToolCallState { + ///When a person allowed the call. + #[serde(skip_serializing_if = "::std::option::Option::is_none")] + pub approved_at: ::std::option::Option<::std::string::String>, + ///From started_at to this change; on tool_call.finished, how long the call took. + pub duration_ms: i64, + ///Why a failed call failed, or what a declined one shows, such as "Not allowed in time". + #[serde(skip_serializing_if = "::std::option::Option::is_none")] + pub error: ::std::option::Option<::std::string::String>, + ///The last thing said about the call while it runs, such as "Searching for flights": a tool_update, the connector binding's pre_speech, or the question it waits on. + #[serde(skip_serializing_if = "::std::option::Option::is_none")] + pub progress: ::std::option::Option<::std::string::String>, + ///When a person declined the call, or its wait for them passed. + #[serde(skip_serializing_if = "::std::option::Option::is_none")] + pub rejected_at: ::std::option::Option<::std::string::String>, + pub started_at: ::std::string::String, + pub status: ToolCallStatus, + pub tool: ::std::string::String, + ///The call, as tool_call, tool_started and tool_ran name it. + pub tool_call_id: ::std::string::String, + pub turn_id: ::std::string::String, + #[serde(rename = "type")] + pub type_: ToolCallFrameType, +} +///in_progress while it runs, waiting while it is held for a person's approval, and success, failed or cancelled once it finished. A call declined, or whose approval nobody gave in time, is cancelled with rejected_at. +#[derive( + ::serde::Deserialize, ::serde::Serialize, Clone, Debug, Eq, Hash, Ord, PartialEq, PartialOrd, +)] +pub enum ToolCallStatus { + #[serde(rename = "in_progress")] + InProgress, + #[serde(rename = "waiting")] + Waiting, + #[serde(rename = "success")] + Success, + #[serde(rename = "failed")] + Failed, + #[serde(rename = "cancelled")] + Cancelled, + /// A value this SDK's spec does not list, as the router sent it. + #[serde(untagged)] + Unlisted(::std::string::String), +} +impl ::std::fmt::Display for ToolCallStatus { + fn fmt(&self, f: &mut ::std::fmt::Formatter<'_>) -> ::std::fmt::Result { + match *self { + Self::InProgress => f.write_str("in_progress"), + Self::Waiting => f.write_str("waiting"), + Self::Success => f.write_str("success"), + Self::Failed => f.write_str("failed"), + Self::Cancelled => f.write_str("cancelled"), + Self::Unlisted(ref value) => f.write_str(value), + } + } +} +impl ::std::str::FromStr for ToolCallStatus { + type Err = self::error::ConversionError; + fn from_str(value: &str) -> ::std::result::Result { + match value { + "in_progress" => Ok(Self::InProgress), + "waiting" => Ok(Self::Waiting), + "success" => Ok(Self::Success), + "failed" => Ok(Self::Failed), + "cancelled" => Ok(Self::Cancelled), + _ => Err("invalid value".into()), + } + } +} +impl ::std::convert::TryFrom<&str> for ToolCallStatus { + type Error = self::error::ConversionError; + fn try_from(value: &str) -> ::std::result::Result { + value.parse() + } +} +impl ::std::convert::TryFrom<::std::string::String> for ToolCallStatus { + type Error = self::error::ConversionError; + fn try_from( + value: ::std::string::String, + ) -> ::std::result::Result { + value.parse() + } +} +///A named set of functions an SDK provides that the agent may offer, how they are loaded and how long what they were given and gave back is kept. A function an SDK registers, declares on a session or hosts in a worker is offered only when a group names it. +#[derive(::serde::Deserialize, ::serde::Serialize, Clone, Debug, PartialEq, Default)] +pub struct ToolGroup { + #[serde(skip_serializing_if = "::std::option::Option::is_none")] + pub loading: ::std::option::Option, + ///What the group is called, unique within the config. + pub name: ::std::string::String, + ///How long the agent's sessions keep what the group's tools were given and gave back on the turns they record: a tool's arguments, its result and the wording of its error. none records none of it; a duration such as 12h or 30d (at least 1h, at most 3650d) keeps it that long after the call and then takes it out, within minutes. The item stays, with its tool name, its time, its call id and whether it failed. Left out or empty, it is kept as long as the session. The app's and the organization's policies (tool_output_retention) can shorten it and never lengthen it. It applies to tool calls recorded after it is set. + #[serde(skip_serializing_if = "::std::option::Option::is_none")] + pub output_retention: ::std::option::Option<::std::string::String>, + ///The functions' names, as the SDK registers them. A name sits in one group at most. + pub tools: ::std::vec::Vec<::std::string::String>, +} +///How a tool is offered to the model. full sends its whole description and input schema from the start. summary sends the first line of its description and its arguments without their descriptions, and the first call returns the full description and schema instead of running it: one more model turn for each tool a conversation uses, for less context on every turn. deferred offers nothing until the model needs it: an agent with a deferred tool is offered search_tool, which finds deferred tools by what they do, and load_tool, which returns their full descriptions and offers them from the model's next step on. +#[derive( + ::serde::Deserialize, ::serde::Serialize, Clone, Debug, Eq, Hash, Ord, PartialEq, PartialOrd, +)] +pub enum ToolLoading { + #[serde(rename = "full")] + Full, + #[serde(rename = "summary")] + Summary, + #[serde(rename = "deferred")] + Deferred, + /// A value this SDK's spec does not list, as the router sent it. + #[serde(untagged)] + Unlisted(::std::string::String), +} +impl ::std::fmt::Display for ToolLoading { + fn fmt(&self, f: &mut ::std::fmt::Formatter<'_>) -> ::std::fmt::Result { + match *self { + Self::Full => f.write_str("full"), + Self::Summary => f.write_str("summary"), + Self::Deferred => f.write_str("deferred"), + Self::Unlisted(ref value) => f.write_str(value), + } + } +} +impl ::std::str::FromStr for ToolLoading { + type Err = self::error::ConversionError; + fn from_str(value: &str) -> ::std::result::Result { + match value { + "full" => Ok(Self::Full), + "summary" => Ok(Self::Summary), + "deferred" => Ok(Self::Deferred), + _ => Err("invalid value".into()), + } + } +} +impl ::std::convert::TryFrom<&str> for ToolLoading { + type Error = self::error::ConversionError; + fn try_from(value: &str) -> ::std::result::Result { + value.parse() + } +} +impl ::std::convert::TryFrom<::std::string::String> for ToolLoading { + type Error = self::error::ConversionError; + fn try_from( + value: ::std::string::String, + ) -> ::std::result::Result { + value.parse() + } +} ///`ToolResultCommand` #[derive(::serde::Deserialize, ::serde::Serialize, Clone, Debug, PartialEq)] pub struct ToolResultCommand { @@ -13605,11 +13808,27 @@ pub struct ToolStats { pub avg_ms: ::std::option::Option, ///What a call put in front of the model, its arguments and its result, estimated at four characters a token. pub avg_tokens: f64, + ///Turns a session took before its first call, on average: 0 is a call on the first turn. Null on the row over every tool. + #[serde(deserialize_with = "::std::option::Option::deserialize")] + pub avg_turns_before_use: ::std::option::Option, pub calls: i64, + ///What the tool's whole definition, its name, description and schema, costs every turn that carries it, estimated at four characters a token. Null when no call recorded it. + #[serde(deserialize_with = "::std::option::Option::deserialize")] + pub definition_tokens: ::std::option::Option, ///Calls that ended in an error. pub failed: i64, + ///How the latest call had the tool offered: full, summary or deferred. Empty when no call recorded it, as calls made before it was recorded and built-in tools. + pub loading: ::std::string::String, #[serde(deserialize_with = "::std::option::Option::deserialize")] pub p99_ms: ::std::option::Option, + ///Sessions that called it, out of the window's sessions, from 0 to 1. Null when no session took a turn. + #[serde(deserialize_with = "::std::option::Option::deserialize")] + pub session_share: ::std::option::Option, + ///Sessions that called it at least once. + pub sessions: i64, + ///What its definition costs offered as a summary. Null unless its loading is summary. + #[serde(deserialize_with = "::std::option::Option::deserialize")] + pub summary_tokens: ::std::option::Option, pub tool: ::std::string::String, } ///Something the recording named, for the providers that pick them out. diff --git a/sdks/rust/tests/client.rs b/sdks/rust/tests/client.rs index 999499b557..2e21bd5ea6 100644 --- a/sdks/rust/tests/client.rs +++ b/sdks/rust/tests/client.rs @@ -5,6 +5,21 @@ use serde_json::json; use support::{SECRET, Server, claims, refusal}; use vision_agents::{Client, ClientOptions, Error, ListResponsesQuery, RouterFailure, types}; +#[tokio::test] +async fn a_client_acting_for_a_user_shares_the_apps_tools() { + let server = Server::start().await; + let backend = server.backend(); + backend + .tools() + .register("lookup", "Look it up", json!({}), |_| async { + Ok::<_, String>("found") + }); + + let user = backend.as_user("ada", "ada-token").unwrap(); + + assert_eq!(user.tools().declared()[0].name, "lookup"); +} + #[tokio::test] async fn a_router_with_nothing_in_front_is_told_the_customer() { let server = Server::start().await; diff --git a/sdks/rust/tests/dispatch.rs b/sdks/rust/tests/dispatch.rs index 29216fdbb5..0116cf7eba 100644 --- a/sdks/rust/tests/dispatch.rs +++ b/sdks/rust/tests/dispatch.rs @@ -9,13 +9,12 @@ use serde_json::json; use support::{Server, claims, response, session}; use tokio::sync::mpsc; use vision_agents::{ - Agent, AgentRef, Client, Dispatch, Error, InboundCall, InboundImage, InboundMessage, - ToolCallContext, types, + Agent, Client, Dispatch, Error, InboundCall, InboundImage, InboundMessage, ToolCallContext, + types, }; -fn stream_support(client: &Client) -> AgentRef { - let agent = client.agent("stream-support"); - agent.tools.register( +fn stream_support(client: Client) -> Client { + client.tools().register( "investigate_sdk", "Read SDK source", json!({"type": "object", "properties": {"sdk": {"type": "string"}}}), @@ -26,7 +25,7 @@ fn stream_support(client: &Client) -> AgentRef { Ok::<_, String>(format!("read {}", arguments["sdk"].as_str().unwrap_or(""))) }, ); - agent + client } #[tokio::test] @@ -34,7 +33,7 @@ async fn a_hosted_function_is_declared_and_answered_over_the_dispatch_socket() { let server = Server::start().await; let dispatch = Dispatch::new(server.client()); dispatch.host( - &stream_support(&server.client()), + &stream_support(server.client()), Some(Duration::from_secs(60)), ); @@ -51,7 +50,7 @@ async fn a_hosted_function_is_declared_and_answered_over_the_dispatch_socket() { socket.expect("host_tools").await, json!({ "type": "host_tools", - "agent_id": "stream-support", + "every_agent": true, "tools": [{ "name": "investigate_sdk", "description": "Read SDK source", @@ -61,9 +60,7 @@ async fn a_hosted_function_is_declared_and_answered_over_the_dispatch_socket() { }) ); socket - .send( - json!({"type": "hosting", "agent_id": "stream-support", "tools": ["investigate_sdk"]}), - ) + .send(json!({"type": "hosting", "every_agent": true, "tools": ["investigate_sdk"]})) .await; socket @@ -102,7 +99,7 @@ async fn a_hosted_function_is_declared_and_answered_over_the_dispatch_socket() { async fn a_worker_tells_the_router_what_it_hosts_every_time_it_is_ready() { let server = Server::start().await; let dispatch = Dispatch::new(server.client()); - dispatch.host(&stream_support(&server.client()), None); + dispatch.host(&stream_support(server.client()), None); let running = { let dispatch = dispatch.clone(); @@ -114,7 +111,7 @@ async fn a_worker_tells_the_router_what_it_hosts_every_time_it_is_ready() { .send(json!({"type": "ready", "worker_id": worker})) .await; let declared = socket.expect("host_tools").await; - assert_eq!(declared["agent_id"], "stream-support"); + assert_eq!(declared["every_agent"], true); assert_eq!(declared["timeout_ms"], 0); } @@ -126,7 +123,7 @@ async fn a_worker_tells_the_router_what_it_hosts_every_time_it_is_ready() { async fn a_worker_whose_tools_are_refused_stops_waiting() { let server = Server::start().await; let dispatch = Dispatch::new(server.client()); - dispatch.host(&stream_support(&server.client()), None); + dispatch.host(&stream_support(server.client()), None); let running = { let dispatch = dispatch.clone(); @@ -134,17 +131,38 @@ async fn a_worker_whose_tools_are_refused_stops_waiting() { }; let mut socket = server.accept().await; socket - .send(json!({"type": "hosting_refused", "agent_id": "stream-support", "reason": "hosting no tools is not hosting"})) + .send(json!({"type": "hosting_refused", "reason": "hosting no tools is not hosting"})) .await; let refused = running.await.unwrap().unwrap_err(); assert!(matches!(refused, Error::Failed { .. })); assert_eq!( refused.to_string(), - "dispatch: the router refused to host tools for agent stream-support: hosting no tools is not hosting" + "dispatch: the router refused to host tools: hosting no tools is not hosting" ); } +#[tokio::test] +async fn run_dispatch_runs_the_clients_tools_for_every_agent() { + let server = Server::start().await; + let client = stream_support(server.client()); + + let running = tokio::spawn(async move { client.run_dispatch().await }); + let mut socket = server.accept().await; + assert_eq!(socket.query, "capacity=4&active=0&handles="); + socket + .send(json!({"type": "ready", "worker_id": "w-1"})) + .await; + let declared = socket.expect("host_tools").await; + assert_eq!(declared["every_agent"], true); + assert_eq!(declared["tools"][0]["name"], "investigate_sdk"); + socket + .send(json!({"type": "tool_call", "id": "call-1", "session_id": "s", "name": "investigate_sdk", "arguments": r#"{"sdk":"ios"}"#})) + .await; + assert_eq!(socket.expect("tool_result").await["output"], "read ios"); + socket.close().await; + running.await.unwrap().unwrap(); +} #[tokio::test] async fn a_worker_answers_calls_and_tells_the_router_how_each_went() { let server = Server::start().await; @@ -595,9 +613,9 @@ async fn hosting_payments( support::Accepted, serde_json::Value, ) { - let agent = server.client().agent("payments"); - agent - .tools + let client = server.client(); + client + .tools() .register_with_tool_call_context( types::SessionTool { name: "pay".into(), @@ -619,8 +637,8 @@ async fn hosting_payments( }, ) .unwrap(); - agent - .tools + client + .tools() .register_with_tool_call_context( types::SessionTool { name: "slow".into(), @@ -635,13 +653,13 @@ async fn hosting_payments( }, ) .unwrap(); - agent - .tools + client + .tools() .register("fast", "Does not", json!({}), |_| async { Ok::<_, String>("done") }); let dispatch = Dispatch::new(server.client()); - dispatch.host(&agent, None); + dispatch.host(&client, None); let running = { let dispatch = dispatch.clone(); tokio::spawn(async move { dispatch.run().await }) diff --git a/sdks/rust/tests/session.rs b/sdks/rust/tests/session.rs index be7ea2471d..97133545a1 100644 --- a/sdks/rust/tests/session.rs +++ b/sdks/rust/tests/session.rs @@ -1102,17 +1102,11 @@ fn holding() -> Tools { } #[tokio::test] -async fn a_function_is_told_who_its_call_answers_and_can_say_how_it_is_going() { +async fn a_function_is_told_who_its_call_answers() { let server = Server::start().await; let tools = Tools::new(); tools .register_with_tool_call_context(tool("book"), |tool_call_context: ToolCallContext, _| async move { - tool_call_context.update("Looking at flights") - .await - .map_err(|error| error.to_string())?; - tool_call_context.say("One moment.") - .await - .map_err(|error| error.to_string())?; Ok::<_, String>(json!({ "id": tool_call_context.id, "session": tool_call_context.session_id, "turn": tool_call_context.turn_id, "user": [tool_call_context.user.id, tool_call_context.user.name], "tier": tool_call_context.user.custom["tier"], @@ -1131,14 +1125,6 @@ async fn a_function_is_told_who_its_call_answers_and_can_say_how_it_is_going() { ) .await; - assert_eq!( - socket.next().await.unwrap(), - json!({"type": "tool_update", "tool_call_id": "t1", "text": "Looking at flights", "verbatim": false}) - ); - assert_eq!( - socket.next().await.unwrap(), - json!({"type": "tool_update", "tool_call_id": "t1", "text": "One moment.", "verbatim": true}) - ); let output: serde_json::Value = serde_json::from_str( socket.expect("tool_result").await["output"] .as_str() diff --git a/sdks/swift/core/Sources/StreamAgents/AgentEvent.swift b/sdks/swift/core/Sources/StreamAgents/AgentEvent.swift index 4d7de2c534..0a9e76e4d3 100644 --- a/sdks/swift/core/Sources/StreamAgents/AgentEvent.swift +++ b/sdks/swift/core/Sources/StreamAgents/AgentEvent.swift @@ -58,9 +58,13 @@ public struct AgentEvent: Sendable, Hashable { case taskCancelled = "task_cancelled" case toolCall = "tool_call" case toolCancel = "tool_cancel" + case toolHeld = "tool_held" case toolStarted = "tool_started" case toolUpdated = "tool_updated" case toolRan = "tool_ran" + case toolCallStarted = "tool_call.started" + case toolCallProgress = "tool_call.progress" + case toolCallFinished = "tool_call.finished" case transferred case pressed case lookedUp = "looked_up" @@ -112,7 +116,9 @@ extension AgentEvent { name: string("name"), arguments: string("arguments"), requestID: string("request_id"), - turnID: string("turn_id")) + turnID: string("turn_id"), + run: self["run"].boolValue ?? true, + approved: self["approved"].boolValue ?? false) } /// A request from the model to run one of the caller's functions. @@ -124,6 +130,11 @@ extension AgentEvent { /// The request and turn that asked, which its result must name again. public var requestID = "" public var turnID = "" + /// Whether this device is the one to run it. Every watcher is sent every call, and + /// only the one the router names answers it. + public var run = true + /// Whether the person already said yes to it, on a voice call, so nobody asks again. + public var approved = false /// The arguments decoded, or an empty dictionary if the model wrote something else. public var argumentValues: [String: RawJSON] { @@ -135,6 +146,66 @@ extension AgentEvent { } } +extension AgentEvent { + /// The call's whole state on a `tool_call.started`, `tool_call.progress` or + /// `tool_call.finished`, or nil for any other event. + public var toolCallState: ToolCallState? { + guard kind == .toolCallStarted || kind == .toolCallProgress || kind == .toolCallFinished, + !string("tool_call_id").isEmpty + else { return nil } + let dates = RouterDates() + let date = { (key: String) in try? dates.decode(self.string(key)) } + return ToolCallState( + id: string("tool_call_id"), + tool: string("tool"), + turnID: string("turn_id"), + status: ToolCallState.Status(rawValue: string("status")), + progress: string("progress"), + startedAt: date("started_at"), + durationMS: Int(self["duration_ms"].numberValue ?? 0), + approvedAt: date("approved_at"), + rejectedAt: date("rejected_at"), + error: string("error")) + } +} + +/// One tool call as the session last reported it, whichever process runs it. +public struct ToolCallState: Sendable, Hashable, Identifiable { + /// The call's `tool_call_id`. + public let id: String + public let tool: String + public let turnID: String + public let status: Status + /// The last thing said about the call while it runs, such as "Searching for flights", or + /// the question it waits on. Empty for none. + public let progress: String + public let startedAt: Date? + /// From `startedAt` to this report; once finished, how long the call took. + public let durationMS: Int + /// When a person allowed the call. + public let approvedAt: Date? + /// When a person declined the call, or its wait for them passed. + public let rejectedAt: Date? + /// Why a failed call failed, or what a declined one shows. Empty for none. + public let error: String + + /// Where a call stands. A value the router added after this SDK shipped is kept as it came. + public struct Status: RawRepresentable, Sendable, Hashable { + public let rawValue: String + public init(rawValue: String) { self.rawValue = rawValue } + + public static let inProgress = Status(rawValue: "in_progress") + /// Held for a person's approval. + public static let waiting = Status(rawValue: "waiting") + public static let success = Status(rawValue: "success") + public static let failed = Status(rawValue: "failed") + public static let cancelled = Status(rawValue: "cancelled") + + /// Whether the call is over. + public var isFinished: Bool { self == .success || self == .failed || self == .cancelled } + } +} + extension AgentEvent { /// A connector tool call the provider refused because the person's own connection lacks /// access it asked for, and the step-up consent the router began for it, or nil when this diff --git a/sdks/swift/core/Sources/StreamAgents/AgentSession.swift b/sdks/swift/core/Sources/StreamAgents/AgentSession.swift index 43de85e5b4..c0430a66fa 100644 --- a/sdks/swift/core/Sources/StreamAgents/AgentSession.swift +++ b/sdks/swift/core/Sources/StreamAgents/AgentSession.swift @@ -28,6 +28,10 @@ public final class AgentSession: ObservableObject { /// first. A conversation kept in Stream Chat also shows each on its reply's step. @Published public private(set) var approvals: [ToolApprovalRequest] = [] + /// Every tool call of the session by its id, as it was last reported: running, waiting for + /// a person's approval, or how it ended, whichever process runs it. Finished calls stay. + @Published public private(set) var toolCalls: [String: ToolCallState] = [:] + /// What the router holds this session by, which is what addresses it and its socket. public var id: String { session.id } @@ -59,6 +63,9 @@ public final class AgentSession: ObservableObject { /// Whoever is reading `events()`, each with a stream of their own, or nil once the session /// has stopped and there is nothing more to read. private var subscribers: [UUID: AsyncStream.Continuation]? = [:] + /// What `on(_:perform:)` registered, locked rather than main-actor isolated so a + /// cancellable released anywhere takes its handler away at once. + private let handlers = AllocatedUnfairLock<[UUID: (kind: AgentEvent.Kind, handle: @MainActor (AgentEvent) -> Void)]>([:]) /// The frames this device's camera showed lately, while it is on a call with it on, which /// answer the router's `get_video_frames`. var cameraFrames: CameraFrames? @@ -125,6 +132,14 @@ public final class AgentSession: ObservableObject { return stream } + /// Calls `handler` with every event of `kind`, such as `.toolCallFinished`, until the + /// returned value is cancelled or released. It needs no loop over `events()`. + public func on(_ kind: AgentEvent.Kind, perform handler: @escaping @MainActor (AgentEvent) -> Void) -> AnyCancellable { + let id = UUID() + handlers.withLock { $0[id] = (kind, handler) } + return AnyCancellable { [handlers] in handlers.withLock { $0[id] = nil } } + } + /// Speaks this without going through the model. `interrupt` stops what the agent is saying /// first, rather than waiting for it to finish. public func say(_ text: String, interrupt: Bool = false) async throws { @@ -354,6 +369,14 @@ public final class AgentSession: ObservableObject { private func apply(_ event: AgentEvent) { conversation.apply(event) + if let state = event.toolCallState { + toolCalls[state.id] = state + } + if let kind = event.kind { + for handler in handlers.withLock({ $0.values.filter { $0.kind == kind }.map(\.handle) }) { + handler(event) + } + } for (id, continuation) in subscribers ?? [:] { switch continuation.yield(event) { case .enqueued: @@ -379,11 +402,12 @@ public final class AgentSession: ObservableObject { private func answer(_ call: AgentEvent.ToolCall) { // Session events are broadcast to observers as well as tool owners. A Python // video worker may own this request; an observer must not resolve it first. + guard call.run else { return } if call.name == CameraFrames.toolName, let cameraFrames { return show(cameraFrames, for: call) } guard let tool = tools[call.name] else { return } - if let approval = tool.approval { + if let approval = tool.approval, !call.approved { approvals.append(ToolApprovalRequest(call: call, approval: approval, tool: tool)) } else { run(call, with: tool) diff --git a/sdks/swift/core/Tests/StreamAgentsTests/ToolCallStateTests.swift b/sdks/swift/core/Tests/StreamAgentsTests/ToolCallStateTests.swift new file mode 100644 index 0000000000..0455daa088 --- /dev/null +++ b/sdks/swift/core/Tests/StreamAgentsTests/ToolCallStateTests.swift @@ -0,0 +1,161 @@ +import Combine +import Foundation +import Network +import Testing + +@testable import StreamAgents + +@MainActor +@Suite struct ToolCallStateTests { + private static let session = Session( + .init( + agentId: "test", callId: "", callType: "agent", createdAt: Date(), id: "test", + modality: .text, state: .live, text: true, userId: "test")) + + private static func state(_ type: String, _ status: String, _ extra: String = "") -> String { + #"{"type":"\#(type)","tool_call_id":"call-1","tool":"search_flights","turn_id":"t1","status":"\#(status)","started_at":"2026-11-18T12:00:00Z","duration_ms":300\#(extra)}"# + } + + /// What a handler heard, kept off the stack because a main-actor closure may not mutate a + /// local it captured. + @MainActor private final class Heard { + var progress: [String] = [] + var finished = 0 + var cancellable: AnyCancellable? + } + + @Test func theSessionKeepsEachCallsLastStateAndAHandlerHearsItsKindUntilCancelled() async throws { + let server = try FrameServer(frames: [ + Self.state("tool_call.started", "in_progress"), + Self.state("tool_call.progress", "in_progress", #","progress":"Searching for flights""#), + Self.state("tool_call.progress", "in_progress", #","progress":"Comparing prices""#), + Self.state("tool_call.finished", "failed", #","error":"the airline is down""#), + ]) + defer { server.listener.cancel() } + let url = try #require(try await server.addresses.first(where: { @Sendable _ in true })) + let session = AgentSession(backend: Backend(url: url, customerID: "test"), session: Self.session, tools: []) + let heard = Heard() + heard.cancellable = session.on(.toolCallProgress) { event in + heard.progress.append(event.string("progress")) + heard.cancellable?.cancel() + } + let finished = session.on(.toolCallFinished) { _ in heard.finished += 1 } + await session.start() + defer { Task { await session.close() } } + + let deadline = ContinuousClock.now + .seconds(30) + while session.toolCalls["call-1"]?.status.isFinished != true, ContinuousClock.now < deadline { + try await Task.sleep(for: .milliseconds(10)) + } + + let call = try #require(session.toolCalls["call-1"]) + #expect(call.status == .failed) + #expect(call.error == "the airline is down") + #expect(call.progress == "", "the finished frame carries the whole state") + #expect(call.durationMS == 300) + #expect(heard.progress == ["Searching for flights"]) + #expect(heard.finished == 1) + finished.cancel() + } + + @Test func aCallForAnotherWatcherIsNotRunAndOneAlreadyApprovedIsNotAskedAgain() async throws { + let server = try FrameServer(frames: [ + #"{"type":"tool_call","id":"elsewhere","name":"lookup","arguments":"{}","run":false}"#, + #"{"type":"tool_call","id":"approved","name":"issue_refund","arguments":"{}","approved":true}"#, + #"{"type":"tool_call","id":"here","name":"lookup","arguments":"{}"}"#, + ]) + defer { server.listener.cancel() } + let url = try #require(try await server.addresses.first(where: { @Sendable _ in true })) + let lookup = AgentTool(name: "lookup", description: "Look it up") { _ in "found" } + let refund = AgentTool( + name: "issue_refund", description: "Refund an order", approval: .init(title: "Refund it?") + ) { _ in "refunded" } + let session = AgentSession( + backend: Backend(url: url, customerID: "test"), session: Self.session, tools: [lookup, refund]) + await session.start() + defer { Task { await session.close() } } + + var answered: Set = [] + for try await data in server.commands { + let command = try JSONDecoder().decode([String: RawJSON].self, from: data) + answered.insert(command["tool_call_id"]?.stringValue ?? "") + if answered.isSuperset(of: ["here", "approved"]) { break } + } + + #expect(answered == ["here", "approved"]) + #expect(session.approvals.isEmpty) + } +} + +/// A real WebSocket peer that sends its frames once the session connects and hands over every +/// frame the session sends back. +private struct FrameServer { + let listener: NWListener + let addresses: AsyncThrowingStream + let commands: AsyncThrowingStream + + init(frames: [String]) throws { + let queue = DispatchQueue(label: "frame-server") + let webSocket = NWProtocolWebSocket.Options() + webSocket.autoReplyPing = true + webSocket.setClientRequestHandler(queue) { _, _ in + .init(status: .accept, subprotocol: nil) + } + let parameters = NWParameters.tcp + parameters.requiredLocalEndpoint = .hostPort(host: "127.0.0.1", port: .any) + parameters.defaultProtocolStack.applicationProtocols.insert(webSocket, at: 0) + let listener = try NWListener(using: parameters) + self.listener = listener + let (addresses, address) = AsyncThrowingStream.makeStream() + let (commands, command) = AsyncThrowingStream.makeStream() + self.addresses = addresses + self.commands = commands + listener.stateUpdateHandler = { state in + switch state { + case .ready: + if let port = listener.port { + address.yield(URL(string: "http://127.0.0.1:\(port.rawValue)")!) + address.finish() + } + case .failed(let error): + address.finish(throwing: error) + command.finish(throwing: error) + default: break + } + } + listener.newConnectionHandler = { connection in + @Sendable func receive() { + connection.receiveMessage { data, _, _, error in + if let data { command.yield(data) } + if let error { + command.finish(throwing: error) + } else { + receive() + } + } + } + connection.stateUpdateHandler = { state in + guard case .ready = state else { return } + for (index, frame) in frames.enumerated() { + let context = NWConnection.ContentContext( + identifier: "frame-\(index)", + metadata: [NWProtocolWebSocket.Metadata(opcode: .text)]) + connection.send( + content: Data(frame.utf8), contentContext: context, + completion: .contentProcessed { error in + if let error { command.finish(throwing: error) } + }) + } + receive() + } + connection.start(queue: queue) + queue.asyncAfter(deadline: .now() + 30) { connection.cancel() } + } + queue.asyncAfter(deadline: .now() + 30) { + let error = AgentsError.unreadable("test socket timed out") + address.finish(throwing: error) + command.finish(throwing: error) + } + listener.start(queue: queue) + } +} diff --git a/tests/test_tool_call.py b/tests/test_tool_call.py index 128acb3bfd..879a3c58eb 100644 --- a/tests/test_tool_call.py +++ b/tests/test_tool_call.py @@ -8,15 +8,11 @@ class Line: - """A line that records what a call sent and answers holds as it is told.""" + """A line that answers holds as it is told.""" def __init__(self, holds: bool = True): - self.sent: list[tuple[str, str, bool]] = [] self.holds = holds - async def update(self, call_id: str, text: str, verbatim: bool) -> None: - self.sent.append((call_id, text, verbatim)) - async def hold(self, call_id: str) -> bool: return self.holds @@ -85,23 +81,16 @@ def test_a_policy_that_means_nothing_is_refused(self, options): class TestToolCallContext: async def test_a_function_whose_first_parameter_is_a_tool_call_is_handed_it(self): registry = FunctionRegistry() - line = Line() @registry.register() async def search(tool_call_context: ToolCallContext, city: str) -> str: - await tool_call_context.update("checking") - await tool_call_context.say("One moment.") return f"{tool_call_context.id} {city}" answer = await registry.call_function( - "search", {"city": "Rome"}, ToolCallContext(id="call-1", line=line) + "search", {"city": "Rome"}, ToolCallContext(id="call-1", line=Line()) ) assert answer == "call-1 Rome" - assert line.sent == [ - ("call-1", "checking", False), - ("call-1", "One moment.", True), - ] def test_the_model_is_not_told_about_the_tool_call(self): registry = FunctionRegistry() @@ -132,7 +121,6 @@ async def test_without_a_call_there_is_an_empty_one_that_does_nothing(self): @registry.register() async def search(tool_call_context: ToolCallContext) -> str: - await tool_call_context.update("still going") await tool_call_context.disallow_interruptions() seen.append(tool_call_context) return "ok"