feat: createPolicyDecider and getDecisionRequests

.changeset/policy-decider-helpers.md

@@ -0,0 +1,7 @@
+---
+"@statelyai/agent": minor
+---
+
+**`createPolicyDecider` runs a policy machine as a decide executor.** Each decision request starts one policy run, whose output event is the choice. The input defaults to the request's `metadata.facts`, offered events, `system`, and `prompt`; the policy's token usage is reported on the result. A run that does not end `done` with an event throws `AgentError` with code `policy-not-done`. With a `store`, each decision's policy run gets its own thread (`policy:<callKey>:<SHA-256 of the input>` by default, or `threadId: (request, info) => ...`). The input must be JSON: the default drops `undefined` keys, and anything else that is not JSON throws before the run.
+
+**`getDecisionRequests` reads the decisions a run asked for from its trace.** From `@statelyai/agent/testing`: `getDecisionRequests(trace, { name? })` returns each decision's first-attempt request with the states it was asked in, from a live or serialized trace, for eval rows and policy probes.

docs/decisions.md

@@ -256,6 +256,29 @@ const { decide } = createAiSdkExecutors({
 
 See [Decision models](models-and-providers.md#decision-models).
 
+## A policy as the decider
+
+A decision can be answered by another machine: a policy that applies a rule when the facts are enough and asks a model only for the rest. `createPolicyDecider` turns a policy machine into a decide executor.
+
+```ts no-check
+import { createPolicyDecider, runAgent } from "@statelyai/agent";
+
+const decide = createPolicyDecider(policyMachine, { executors: modelExecutors });
+
+await runAgent(deskMachine, { input, executors: { ...modelExecutors, decide } });
+```
+
+- Each decision request starts one run of the policy. Its output, an event, is the choice. The environment still judges it: an event it does not accept is retried like any decider's.
+- The policy's input defaults to `{ facts, options, system, prompt }`: the request's `metadata.facts`, the offered events, and its prompt. The policy's input schema validates it. Pass `input: (request) => ...` to build it another way.
+- The policy's own `agent.decide` invokes are answered by `executors`. Pass the environment's `metadata` on to them, so the model sees what it would without the policy.
+- The result carries the policy's token usage, and none when it decided with a rule.
+- A run that does not end `done` with an event throws `AgentError` with code `policy-not-done`. Give each of the policy's decisions an `onError` that falls back to an event the environment always accepts.
+- Every other `runAgent` option passes through: `store` and `metadata` to journal the policy's runs, `onTrace` to trace them.
+- Each decision's policy run gets its own `store` thread, `policy:${info.callKey}:${hash}` by default, where `hash` is the SHA-256 of the policy input: a thread that already has entries is resumed, so a shared one would return an earlier choice. The hash keeps a fork that asks something else at the same call site off its parent's thread. Pass `threadId: (request, info) => ...` to name them yourself; it is required with a `store` when the decider is called without a `callKey`.
+- The policy's input must be JSON, since it opens the run's event log. The default drops keys whose value is `undefined`; any other value that is not JSON throws before the run.
+
+To route by request name, wrap it: `decide: (request, info) => (request.name === "chooseResolution" ? policyDecide(request, info) : modelExecutors.decide!(request, info))`, where `policyDecide` is the executor `createPolicyDecider` returned.
+
 ## Decisions at the boundary of a plain machine
 
 A machine does not have to know about AI. `decideNext` asks a model to pick one event that a plain XState machine accepts right now, built with `createMachine` or `setup`, with no `agent.*` invoke anywhere.

docs/environment-and-policy.md

@@ -30,7 +30,7 @@ environment (TypeScript, fixed)          policy (TS or JSON, improvable)
 | Policy      | A model, reviewed by a person | TypeScript `setupAgent` machine, or JSON `AgentWorkflowConfig` | Between runs, on measured gains | TS: `tsc`, `lintAgentMachine`, replay, objective; JSON: `verifyAgentConfig`, replay, objective |
 
 - The environment exposes decision points: states where it asks for one event. Its guards decide whether to accept whatever the policy proposes, so a policy cannot take an action the environment forbids.
-- The policy answers at those decision points. It can be a single model call, a ladder of `choice` states over the facts, a judgment, or a request to ask the customer. Its final state outputs the event it proposes. The host runs the policy as the environment's `decide` executor, as the [worked example](../examples/self-improving-refunds) does.
+- The policy answers at those decision points. It can be a single model call, a ladder of `choice` states over the facts, a judgment, or a request to ask the customer. Its final state outputs the event it proposes. The host runs the policy as the environment's `decide` executor: `createPolicyDecider(policyMachine, { executors })` starts one policy run per decision and returns its output event. See [A policy as the decider](decisions.md#a-policy-as-the-decider).
 - An invariant sentinel, a state that is entered only when a rule is broken, stays unreachable for every policy. `canReach` proves it once, against the environment, for every event sequence and payload variant you list. See [Payloads and invoke outputs](verify.md#payloads-and-invoke-outputs).
 
 The environment can be a plain XState machine. [`decideNext`](decisions.md#decisions-at-the-boundary-of-a-plain-machine) asks a model to pick one event that a plain XState machine accepts right now, and [closed choices](decisions.md#closed-choices) list the concrete events a decider may pick.

docs/evals.md

@@ -220,6 +220,27 @@ const stats = foldTraces(traces, {
 - `decideNext` called by the host emits no trace. Push its request onto the trace before sending the event: `trace.push({ type: "request.start", request: result.request })`.
 - Rows come in the order states were first seen. Sort them yourself, for example by `failures / visits`.
 
+## Decision requests from traces
+
+`getDecisionRequests` (from `@statelyai/agent/testing`) lists the decisions a run asked for, read from its trace: each decision's first `request.start`, and the states the run was in.
+
+```ts no-check
+import { getDecisionRequests } from "@statelyai/agent/testing";
+
+for (const { states, request } of getDecisionRequests(trace, { name: "chooseResolution" })) {
+  const { event } = await decide(request); // a candidate decider
+  console.log(
+    states,
+    request.options?.map((o) => o.event),
+    event,
+  );
+}
+```
+
+- A trace is one run's events, live or from `serializeTraceEvent`. Retries of a decision are left out, so each decision appears once.
+- A serialized request keeps its `options`, `withheld`, `metadata`, `system`, and `prompt`. Event schemas do not survive serialization, so replay closed choices rather than open ones.
+- Use the requests as eval rows (a decider on each, compared with a label) or as probe inputs for a policy.
+
 ## Parity with the code you replace
 
 `compareRuns` (from `@statelyai/agent/testing`) runs every scenario through the existing implementation and its replacement, and labels each difference. Use it when you port a loop to a machine and need to show where they behave the same and where the machine differs on purpose.

skills/add-evals/SKILL.md

@@ -84,11 +84,20 @@ Use `explorePaths` to assert the set of reachable final states, and `await simul
 - `explorePaths` and `canReach` use `options.payloads` **instead of** the decision's own offered options, so they explore values the machine would never offer. That is what a boundary proof needs.
 - `simulateAgent` keeps the offered options: a scripted event the decision did not offer is refused, and the run ends in the decision's `onError`. Script only offered events, or pick an input that offers the value.
 
-**Decision evals.** A row is a decision request and the event a person says is right. Write rows by hand, or take them from recorded traces: the first attempt of each decision is a `request.start` event with `request.kind === "decision"` and `request.attempts.length === 0`, and its `request` carries `name`, `options`, and `metadata`. Cast the serialized `request` to `AgentExecutorDecisionRequest`, and compare events without caring about key order. If the decider reads anything outside the request (the ticket, the tool history), store that in the row too.
+**Decision evals.** A row is a decision request and the event a person says is right. Write rows by hand, or take them from recorded traces with `getDecisionRequests(trace, { name })` from `@statelyai/agent/testing`: each decision's first attempt, with the states it was asked in, its `options`, and its `metadata`. Compare events without caring about key order. If the decider reads anything outside the request (the ticket, the tool history), store that in the row too.
 
 ```ts
-// Your helper: rows for one decision state, from traces or a fixture file.
-const rows: Array<{ id: string; request: AgentExecutorDecisionRequest; expected: ChosenEvent }> = loadDecisionRows("deciding");
+import { getDecisionRequests } from "@statelyai/agent/testing";
+
+// One row per recorded decision; `expected` is the label a person confirmed for that decision.
+// A scenario can decide more than once, so label each decision by its position, not the scenario alone.
+const rows = traces.flatMap(({ scenario, trace }) =>
+  getDecisionRequests(trace, { name: "chooseResolution" }).map(({ request }, i) => ({
+    id: `${scenario.id}#${i}`,
+    request,
+    expected: labels[`${scenario.id}#${i}`],
+  })),
+);
 
 test.each(rows)("deciding: $id", async ({ request, expected }) => {
   const { event } = await decide(request); // the policy decider, or the model executor

skills/add-traces/SKILL.md

@@ -100,7 +100,7 @@ await writeFile(`runs/${file}.trace.json`, JSON.stringify(trace));
 Decide with the user whether `runs/` is committed or gitignored.
 
 - In production, send them to OpenTelemetry instead: `createOtelTraceHandler({ tracer })` from `@statelyai/agent/otel` is an `onTrace` handler.
-- **A policy run inside a decider** is a separate run. Pass it the same `onTrace` (and its own `store` thread, `<threadId>:policy:<n>`) if you want per-state stats for the policy too.
+- **A policy run inside a decider** is a separate run. Pass it the same `onTrace` (and a `store`) if you want per-state stats for the policy too. `createPolicyDecider` gives each decision's policy run its own thread, `policy:<callKey>`; never give it one fixed `threadId`, or the second decision resumes the first.
 - **A plain XState machine** (built with `createMachine` or `setup`, decided with `decideNext`): drive it with `createAgentRuntime` and `runToQuiescence`, which accept plain machines, and pass each `decideNext` result as `{ event, attempts }` so refused choices reach the log. Push `{ type: "request.start", request: result.request }` onto the trace before sending the event, since a host-driven decision emits no trace itself.
 - **A plain `createActor` run you cannot change:** attach `recordLog(actor)` from `@statelyai/agent/log` before `actor.start()`, and pass `inspect: traceTransitions((event) => trace.push(serializeTraceEvent(event)))` to `createActor`. `recordLog` cannot record decision attempts.
 

skills/improve-machine/SKILL.md

@@ -37,7 +37,7 @@ The rounds are the same for both formats. What differs is what the diff may touc
 **No policy yet.** `loop-to-machine` leaves a single model decision inside the environment when the old code had no policy logic. Then:
 
 - Record `v0` in a new `CHANGELOG.md` next to the environment (in a `policy/` directory): "the model decides directly". This entry goes in the working tree.
-- The first recommended candidate introduces the policy as `v1`: a policy file that reproduces the current behavior (it passes the environment's prompt and facts through to the model) plus the one rule the diagnosis found, and the routing line in the runner (`withPolicy(modelExecutors)`). The pattern is in `../loop-to-machine/policy.md`.
+- The first recommended candidate introduces the policy as `v1`: a policy file that reproduces the current behavior (it passes the environment's prompt and facts through to the model) plus the one rule the diagnosis found, and the routing line in the runner (`withPolicy(modelExecutors)`). The policy runs as the decider through `createPolicyDecider` from `@statelyai/agent`; the pattern is in `../loop-to-machine/policy.md`.
 - Its `description` is the environment's goal with measurable success criteria. If the environment's goal has none, draft them from the confirmed objective and list the draft under questions for the user.
 - If the policy needs a fact the environment does not pass in the decision's `metadata.facts`, that is an environment change request.
 
@@ -152,7 +152,7 @@ Run every check; a failure's output is the repair prompt. Cap repairs at 3.
 1. **Environment unchanged:** every environment file hashes as recorded. A change is never repaired; discard the candidate.
 2. **Types:** `tsc --noEmit` passes.
 3. **Lint:** `lintAgentMachine(policy)` from `@statelyai/agent/testing` reports no errors.
-4. **Paths:** for probe inputs built from recorded decision requests (one per training scenario that reaches the decision: its facts and offered options), `canReach(policy, (snapshot) => snapshot.status === "done" && !isOfferedEvent(snapshot.output), { input })` is `{ reachable: false, complete: true }`: every final output is one of the offered events. Pass `outputs` for any actor the policy invokes, and `errors: { "agent.decide": new Error("down") }` so the fallback path is checked too.
+4. **Paths:** for probe inputs built from recorded decision requests (`getDecisionRequests(trace, { name })` from `@statelyai/agent/testing`, one per training scenario that reaches the decision: its facts and offered options), `canReach(policy, (snapshot) => snapshot.status === "done" && !isOfferedEvent(snapshot.output), { input })` is `{ reachable: false, complete: true }`: every final output is one of the offered events. Pass `outputs` for any actor the policy invokes, and `errors: { "agent.decide": new Error("down") }` so the fallback path is checked too.
 5. **Source rules:** the policy file is still one `setupAgent` machine; its imports from the environment are only schemas, types, constants, and exported guard functions; it names no actor the environment owns. These are source checks: read the diff, or grep its imports and actor names.
 6. **Overfit:** the diff contains no literal from the scenarios (ids, ticket text, customer ids, order ids). A branch that matches one scenario's data is memorization; discard it.
 7. **Existing evals:** the rule and decision evals from `add-evals` still pass. Tests that pin measured numbers of the current version (a baseline failure list, a call count) are expected to change; update them in the same pull request and list each change in its body.

skills/loop-to-machine/policy.md

@@ -32,7 +32,7 @@ export const deskEvents = {
 
 ```ts
 import { z } from "zod";
-import { runAgent, setupAgent, type AgentDecisionExecutor, type AgentRequestExecutors } from "@statelyai/agent";
+import { createPolicyDecider, setupAgent, type AgentDecisionExecutor, type AgentRequestExecutors } from "@statelyai/agent";
 import { deskEvents, factsSchema } from "./desk.js";
 
 /** One proposal: an environment event with its `type`. */
@@ -119,22 +119,13 @@ export const policyMachine = policySetup.createMachine({
   },
 });
 
-/** Runs the policy as the environment's decider. */
+/**
+ * Runs the policy as the environment's decider. createPolicyDecider passes the request's
+ * `metadata.facts`, offered events, `system`, and `prompt` as the policy's input (its input
+ * schema validates them), and returns the policy's output event with its token usage.
+ */
 export function policyDecider(executors: AgentRequestExecutors): AgentDecisionExecutor {
-  return async (request, info) => {
-    const result = await runAgent(policyMachine, {
-      input: {
-        facts: factsSchema.parse(request.metadata?.facts),
-        options: (request.options ?? []).map((option) => proposalSchema.parse(option.event)),
-        system: request.system,
-        prompt: request.prompt,
-      },
-      executors,
-      signal: info?.signal,
-    });
-    if (result.status !== "done") throw new Error(`policy settled '${result.status}'`);
-    return { event: result.output };
-  };
+  return createPolicyDecider(policyMachine, { executors });
 }
 ```
 

src/decision-requests.test.ts

@@ -0,0 +1,90 @@
+import { describe, expect, test } from "vitest";
+import { z } from "zod";
+import { getDecisionRequests } from "./decision-requests.js";
+import type { AgentDecisionExecutor } from "./decision.js";
+import { runAgent, serializeTraceEvent, type AgentTraceEvent } from "./run-agent.js";
+import { setupAgent } from "./setup-agent.js";
+
+const desk = setupAgent({
+  context: z.object({ requested: z.number() }),
+  input: z.object({ requested: z.number() }),
+  events: { REFUND: z.object({ amount: z.number() }), ESCALATE: z.object({}), NOTE: z.object({}) },
+}).createMachine({
+  context: ({ input }) => input,
+  initial: "triaging",
+  states: {
+    triaging: {
+      invoke: {
+        src: "agent.decide",
+        input: { model: "m", name: "triage", allowedEvents: ["NOTE"] },
+      },
+      on: { NOTE: { target: "deciding" } },
+    },
+    deciding: {
+      invoke: {
+        src: "agent.decide",
+        input: ({ context }) => ({
+          model: "m",
+          name: "chooseResolution",
+          allowedEvents: ["REFUND", "ESCALATE"],
+          payloads: { REFUND: [{ amount: context.requested }, { amount: 1000 }] },
+          metadata: { facts: { requested: context.requested } },
+        }),
+      },
+      on: {
+        REFUND: ({ context, event }) =>
+          event.amount <= context.requested ? { target: "refunded" } : undefined,
+        ESCALATE: { target: "escalated" },
+      },
+    },
+    refunded: { type: "final" },
+    escalated: { type: "final" },
+  },
+});
+
+/** Notes, then proposes an illegal refund once before the legal one, so the decision is retried. */
+const decide: AgentDecisionExecutor = async (request) => {
+  if (request.name === "triage") return { event: { type: "NOTE" } };
+  return {
+    event:
+      request.attempts.length === 0 ? { type: "REFUND", amount: 999 } : request.options![0]!.event,
+  };
+};
+
+async function traceOf() {
+  const trace: AgentTraceEvent[] = [];
+  const result = await runAgent(desk, {
+    input: { requested: 40 },
+    executors: { decide },
+    onTrace: (event) => trace.push(event),
+  });
+  expect(result.snapshot.value).toBe("refunded");
+  return trace;
+}
+
+describe("getDecisionRequests", () => {
+  test("lists each decision once, on its first attempt, with the state it was asked in", async () => {
+    const requests = getDecisionRequests(await traceOf());
+
+    expect(requests.map((r) => [r.states, r.request.name, r.request.attempts.length])).toEqual([
+      [["triaging"], "triage", 0],
+      [["deciding"], "chooseResolution", 0],
+    ]);
+  });
+
+  test("reads a serialized trace, keeping the offered options and the facts", async () => {
+    const serialized = JSON.parse(
+      JSON.stringify((await traceOf()).map((e) => serializeTraceEvent(e))),
+    );
+    const [row] = getDecisionRequests(serialized, { name: "chooseResolution" });
+
+    expect(row?.states).toEqual(["deciding"]);
+    expect(row?.request).toMatchObject({
+      name: "chooseResolution",
+      metadata: { facts: { requested: 40 } },
+      options: [{ event: { type: "REFUND", amount: 40 } }, { event: { type: "ESCALATE" } }],
+    });
+    // The guard-refused option is on the request as `withheld`, not `options`.
+    expect(row?.request.withheld).toEqual([{ type: "REFUND", amount: 1000 }]);
+  });
+});