Human-sent transitions, ignored events in traces, MCP workflow example

.changeset/human-actor-and-ignored-trace.md

@@ -0,0 +1,9 @@
+---
+"@statelyai/agent": patch
+---
+
+Events a person sends, and ignored events in traces.
+
+- **`meta: { actor: "human" }` on a transition** means a person sends its event. `agent.decide` and `decideNext` never offer it: a closed choice lists its options in `withheld`, and a choice of it anyway is retried. `getAcceptedEvents(snapshot, { actor: "agent" })` returns the same model-facing list. `describeMachine` marks the transition `(human)`.
+- **New lint check `decide-allows-human-event`** (error): a decision's literal `allowedEvents` names a tagged event. `verifyAgentConfig` fails its `lint` gate on it.
+- **`machine.transition` trace events carry `ignored: true`** for the host event the state had no transition for, the one `result.ignored` reports. `foldTraces` counts them per state as `ignored`.

docs/debugging.md

@@ -24,6 +24,6 @@ Common errors:
 | `AgentMaxModelCallsExceededError` | The run exceeded its configured model-call budget.                                                                                    |
 | XState version error              | The persisted snapshot version needs the machine's native `migrate` function.                                                         |
 
-An event the active state has no transition for is not an error: the run settles normally and the event comes back as `result.ignored`.
+An event the active state has no transition for is not an error: the run settles normally and the event comes back as `result.ignored`. Its `machine.transition` trace event carries `ignored: true`.
 
-`lintAgentMachine` reports only Agent-specific mistakes: decisions with no candidate events, direct object request sources that a host cannot bind, and invokes with no error path. General state-machine lint belongs in XState tooling.
+`lintAgentMachine` reports only Agent-specific mistakes: decisions with no candidate events, decisions that list an event only a person sends, direct object request sources that a host cannot bind, and invokes with no error path. General state-machine lint belongs in XState tooling.

docs/decisions.md

@@ -121,6 +121,32 @@ const request = createDecisionRequest({
 });
 ```
 
+## Events a person sends
+
+Tag a transition with `meta: { actor: "human" }` when only a person may send its event, such as an approval. No model-facing decision offers that event in a state that handles it this way.
+
+```ts no-check
+reviewing: {
+  invoke: {
+    src: "agent.decide",
+    input: { model: "quick", allowedEvents: ["revise"] },
+  },
+  on: {
+    approve: { target: "approved", meta: { actor: "human" } },
+    revise: { target: "revising" },
+  },
+},
+```
+
+- `agent.decide` and `decideNext` drop the event from the candidates. A closed choice lists its options in `withheld`, and a choice of it anyway is retried.
+- `getAcceptedEvents(snapshot, { actor: "agent" })` returns the same model-facing list, for a host that offers events itself.
+- `describeMachine` marks the transition `(human)`.
+- `lintAgentMachine` reports `decide-allows-human-event` when a decision's literal `allowedEvents` names a tagged event. `verifyAgentConfig` fails its `lint` gate on it.
+- Only object transitions can be tagged. A transition written as a function has no static `meta`.
+- A plain `createMachine` declares `schemas: { meta: interactionMetaSchema }` to type the tag; `setupAgent` machines have it by default.
+
+The tag does not change what the machine accepts. A host still sends the event when a person answers, and `parseAgentEvent` still parses it.
+
 ## Validation and retries
 
 <!-- decision validation checks and retry behavior from src/decision.ts -->

docs/environment-and-policy.md

@@ -152,6 +152,7 @@ The `vocabulary` gate keeps a candidate inside the environment's contract: it li
 People own everything that defines "better" and everything the policy cannot touch.
 
 - The environment: its states, guards, invariants, and the `canReach` proofs behind them.
+- The events a person sends. Tag their transitions `meta: { actor: "human" }` so no decision offers them and `verifyAgentConfig` fails a policy that lists them. See [Events a person sends](decisions.md#events-a-person-sends).
 - The goal, its success criteria, and the objective function. A model can draft an objective from the goal; a person confirms it before it is used.
 - The vocabulary a policy may reference.
 - The scenario set, the held-out split, and the regression tolerance.

docs/evals.md

@@ -199,7 +199,7 @@ A queue that runs dry throws an `AgentError` with code `seam-script-exhausted`,
 
 ## Per-state stats
 
-`foldTraces` (from `@statelyai/agent/testing`) folds many runs' traces into one row per state: how many runs visited it, how many of those failed, and how many options the guards withheld at decisions made there.
+`foldTraces` (from `@statelyai/agent/testing`) folds many runs' traces into one row per state: how many runs visited it, how many of those failed, how many options the guards withheld at decisions made there, and how many events it ignored.
 
 ```ts no-check
 import { foldTraces } from "@statelyai/agent/testing";
@@ -211,12 +211,13 @@ const stats = foldTraces(traces, {
     return end?.type !== "run.end" || !end.snapshot.matches("refunded");
   },
 });
-// [{ state: "deciding", visits: 2, failures: 1, withheld: 2 }, ...]
+// [{ state: "deciding", visits: 2, failures: 1, withheld: 2, ignored: 0 }, ...]
 ```
 
 - A trace is one run's events, live or from `serializeTraceEvent`. `failed` judges the whole trace.
 - A run visits each state in a `machine.transition`'s `from.value` and `snapshot.value`. Nested states are dot-joined paths; a parallel state counts once per active leaf.
 - `withheld` comes from a decision's first `request.start`: the length of `request.withheld`, counted toward the state the run was in, which is the `from.value` of the transition the choice causes.
+- `ignored` counts `machine.transition` events with `ignored: true`: a host event the state had no transition for, counted toward that state.
 - `decideNext` called by the host emits no trace. Push its request onto the trace before sending the event: `trace.push({ type: "request.start", request: result.request })`.
 - Rows come in the order states were first seen. Sort them yourself, for example by `failures / visits`.
 

docs/observability.md

@@ -17,7 +17,7 @@ await runToQuiescence(
 
 Agent trace kinds are `run.start`, `request.start`, `request.end`, `request.error`, `stream.chunk`, `machine.transition`, `emit`, `usage.dropped`, and `run.end`.
 
-A `machine.transition` event carries `from.value`, the state value the event arrived in, next to `snapshot`, the state after it. The first transition of a stream (the init event, or a restore on resume) has no earlier state, so its `from.value` equals its `snapshot.value`. `traceTransitions` sets it the same way for a plain actor.
+A `machine.transition` event carries `from.value`, the state value the event arrived in, next to `snapshot`, the state after it. The first transition of a stream (the init event, or a restore on resume) has no earlier state, so its `from.value` equals its `snapshot.value`. `traceTransitions` sets it the same way for a plain actor. When the host's own event had no transition in the active state, the runtime's `machine.transition` also carries `ignored: true` (the event `result.ignored` reports), so evals can count those events from the trace alone.
 
 A `request.end` event carries the call's `output` and `raw`, plus what the executor reported about the call itself: `usage` when it reported tokens, and `finishReason` when it reported why the call stopped, normalized to `'stop'`, `'length'`, `'tool-calls'`, `'content-filter'`, or `'other'`. See [Finish reason and truncation](text-requests.md#finish-reason-and-truncation).
 

docs/verify.md

@@ -40,11 +40,12 @@ skip checks by code.
 assertAgentMachine(machine, { warnings: true });
 ```
 
-| Code                      | Severity | Fires when                                                                                                                                                                                                                                                      |
-| ------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `decide-without-events`   | error    | A state invokes `agent.decide` but neither it nor any ancestor handles any event, so the chosen event can never be delivered.                                                                                                                                   |
-| `invoke-without-on-error` | warning  | An invoke declares no `onError`, and neither its state nor any ancestor handles an actor error, so a rejected request or actor lands the machine in an error state with no modeled recovery. Add `onError` targeting a `failed` final state or a bounded retry. |
-| `direct-object-src`       | warning  | An invoke `src` is a direct object or machine value that `runToQuiescence` cannot rebind, so it inherits no host executors.                                                                                                                                     |
+| Code                        | Severity | Fires when                                                                                                                                                                                                                                                      |
+| --------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `decide-without-events`     | error    | A state invokes `agent.decide` but neither it nor any ancestor handles any event, so the chosen event can never be delivered.                                                                                                                                   |
+| `decide-allows-human-event` | error    | A decision's literal `allowedEvents` names an event the state or an ancestor handles through a transition tagged `meta: { actor: "human" }`, so the model is never offered it. See [Events a person sends](decisions.md#events-a-person-sends).                 |
+| `invoke-without-on-error`   | warning  | An invoke declares no `onError`, and neither its state nor any ancestor handles an actor error, so a rejected request or actor lands the machine in an error state with no modeled recovery. Add `onError` targeting a `failed` final state or a bounded retry. |
+| `direct-object-src`         | warning  | An invoke `src` is a direct object or machine value that `runToQuiescence` cannot rebind, so it inherits no host executors.                                                                                                                                     |
 
 ## Test assertions
 

examples/README.md

@@ -113,6 +113,7 @@ key or a specific runtime, so they set `manual: true` and are not exported from
 - [cloudflare-agent-host](cloudflare-agent-host): a Durable Object whose SQLite event log is the source of truth
 - [cloudflare-workers-ai-host](cloudflare-workers-ai-host): a provider with no native tool calling, so the legal events are serialized into the prompt
 - [next-host](next-host): controlled mode across a stateless Next route handler
+- [mcp-workflow](mcp-workflow): a coding agent drives a bug-fix machine through MCP tools; review events are tagged `actor: "human"`, so only a person sends them
 
 ## Evals and verification
 

examples/mcp-workflow/index.test.ts

@@ -0,0 +1,106 @@
+import { expect, test } from "vitest";
+import { Client } from "@modelcontextprotocol/sdk/client/index.js";
+import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
+import { answer, createBugfixServer, type RunStatus } from "./index.js";
+
+async function connect() {
+  const [clientSide, serverSide] = InMemoryTransport.createLinkedPair();
+  await createBugfixServer().connect(serverSide);
+  const client = new Client({ name: "test", version: "1.0.0" });
+  await client.connect(clientSide);
+  const call = async (name: string, args: Record<string, unknown>) => {
+    const result = (await client.callTool({ name, arguments: args })) as {
+      content: { text: string }[];
+      isError?: boolean;
+    };
+    const text = result.content[0]!.text;
+    return result.isError
+      ? { error: text }
+      : (JSON.parse(text) as RunStatus & { workflow?: string });
+  };
+  return { client, call };
+}
+
+test("lists the three tools", async () => {
+  const { client } = await connect();
+  const { tools } = await client.listTools();
+  expect(tools.map((tool) => tool.name)).toEqual(["start", "status", "send"]);
+});
+
+test("the agent works the run; a person approves it", async () => {
+  const { call } = await connect();
+
+  const started = await call("start", { issue: "Totals are off by one" });
+  if ("error" in started) throw new Error(started.error);
+  expect(started.workflow).toContain("on approve (human) → approved");
+  expect(started.state).toBe("reproducing");
+  expect(started.instructions).toBe("Write a test that fails because of the bug.");
+  expect(started.events.map((event) => event.type)).toEqual(["reproduced", "cannotReproduce"]);
+  expect(started.events[0]!.payload).toMatchObject({ required: ["test"] });
+
+  const { runId } = started;
+  await call("send", { runId, event: { type: "reproduced", test: "totals.test.ts" } });
+  await call("send", { runId, event: { type: "fixed", summary: "Use < instead of <=" } });
+  const inReview = await call("send", { runId, event: { type: "testsPassed" } });
+  expect(inReview).toMatchObject({
+    state: "inReview",
+    waitingFor: "Review the fix: Use < instead of <=",
+    events: [],
+  });
+
+  // Review is a person's call.
+  expect(await call("send", { runId, event: { type: "approve" } })).toEqual({
+    error: "A person sends 'approve'. Call status later to see their answer.",
+  });
+
+  await answer(runId, { type: "approve" });
+  expect(await call("status", { runId })).toMatchObject({
+    state: "approved",
+    output: { summary: "Use < instead of <=" },
+  });
+});
+
+test("a bad payload and an event the state does not handle come back as tool errors", async () => {
+  const { call } = await connect();
+  const started = await call("start", { issue: "Crash on empty input" });
+  if ("error" in started) throw new Error(started.error);
+  const { runId } = started;
+
+  const badPayload = await call("send", { runId, event: { type: "reproduced" } });
+  expect(badPayload).toHaveProperty("error", expect.stringContaining("reproduced"));
+
+  expect(await call("send", { runId, event: { type: "testsPassed" } })).toEqual({
+    error: "'testsPassed' does not apply in state 'reproducing'.",
+  });
+  expect(await call("status", { runId })).toMatchObject({ state: "reproducing" });
+});
+
+test("requested changes send the agent back to fixing", async () => {
+  const { call } = await connect();
+  const started = await call("start", { issue: "Slow search" });
+  if ("error" in started) throw new Error(started.error);
+  const { runId } = started;
+  await call("send", { runId, event: { type: "reproduced", test: "search.test.ts" } });
+  await call("send", { runId, event: { type: "fixed", summary: "Add an index" } });
+  await call("send", { runId, event: { type: "testsPassed" } });
+
+  await answer(runId, { type: "requestChanges", notes: "Cover the empty query too" });
+  expect(await call("status", { runId })).toMatchObject({
+    state: "fixing",
+    events: [{ type: "fixed" }],
+  });
+});
+
+test("concurrent sends to one run apply one at a time", async () => {
+  const { call } = await connect();
+  const started = await call("start", { issue: "Flaky retry" });
+  if ("error" in started) throw new Error(started.error);
+  const { runId } = started;
+
+  const [first, second] = await Promise.all([
+    call("send", { runId, event: { type: "reproduced", test: "retry.test.ts" } }),
+    call("send", { runId, event: { type: "cannotReproduce", notes: "Passes locally" } }),
+  ]);
+  expect(first).toMatchObject({ state: "fixing" });
+  expect(second).toEqual({ error: "'cannotReproduce' does not apply in state 'fixing'." });
+});