Skip to content

Human-sent transitions, ignored events in traces, MCP workflow example - #176

Merged
davidkpiano merged 4 commits into
nextfrom
davidkpiano/referee-additions
Oct 24, 2026
Merged

davidkpiano merged 4 commits into
nextfrom
davidkpiano/referee-additions

Conversation

@davidkpiano

@davidkpiano davidkpiano commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

Three additions for hosts that let an outside agent drive a machine.

Ignored events in traces

  • machine.transition carries ignored: true for the host event the state had no transition for, the same event result.ignored reports. Runtime only; traceTransitions does not set it.
  • foldTraces counts them per state as ignored.

Transitions a person sends

  • meta: { actor: "human" } on a transition marks its event as sent by a person, such as an approval.
  • agent.decide and decideNext never offer it. A closed choice lists its options in withheld; a choice of it anyway is retried.
  • getAcceptedEvents(snapshot, { actor: "agent" }) returns the model-facing list.
  • describeMachine marks the transition (human).
  • New lint check decide-allows-human-event (error): a decision's literal allowedEvents names a tagged event. verifyAgentConfig fails its lint gate on it. Reads fromConfig @expr inputs too.
  • Object transitions only; a function transition has no static meta.
  • Docs: Events a person sends.

examples/mcp-workflow

  • An MCP server (start, status, send) over a bug-fix machine a coding agent works through.
  • Review events are tagged actor: "human", so send refuses them; the host's UI answers with answer(...).
  • Adds @modelcontextprotocol/sdk@^1.29.0 as a dev dependency. An example, not a subpath: Scope leaves MCP transport to the host.

For review

  • Lint severity is error, not warning: listing an event the model can never be offered is always a mistake, and a model-written config should fail the gate.

Verification

  • vitest --run: 137 files, 1624 tests pass.
  • typecheck:src, typecheck:examples, lint, format:check, knip, docs:check pass.
  • Drove the stdio server end to end with an MCP stdio client.

Devin Review

Summary by CodeRabbit

  • New Features
    • Transitions can be marked as human-handled, keeping their events out of agent decision options. Excluded options are reported as withheld where applicable, and machine descriptions identify human-handled transitions.
    • Added an example workflow for managing bug fixes with agent actions and human review.
  • Bug Fixes
    • Linting now flags decisions that allow events handled by human-tagged transitions.
  • Observability
    • Transition traces mark ignored events, and state statistics include counts of those events.

- machine.transition carries ignored: true for the host event result.ignored reports; foldTraces counts it per state
- transition meta actor: "human": agent.decide and decideNext withhold the event, getAcceptedEvents({ actor: "agent" }) drops it, describeMachine marks it, lint reports decide-allows-human-event
@changeset-bot

changeset-bot Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 906b417

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 6 packages
Name Type
@statelyai/agent Patch
@statelyai/agent-demo Patch
@statelyai/example-next-host Patch
@statelyai/example-tanstack-ai-stream Patch
@statelyai/example-cloudflare-agent-host Patch
@statelyai/example-cloudflare-workers-ai-host Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 13 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 2 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 16a4bc5e-e7dc-4e48-b475-1f254c85f0b4
📥 Commits

Reviewing files that changed from the base of the PR and between 1764265 and 906b417.

📒 Files selected for processing: 6
  • examples/mcp-workflow/index.test.ts
  • examples/mcp-workflow/index.ts
  • src/events.ts
  • src/human-actor.test.ts
  • src/steps.ts
  • src/verify.ts

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 909507b9-eebe-4337-bf13-123ba8d9e483
📥 Commits

Reviewing files that changed from the base of the PR and between c620d5c and 1764265.

⛔ Files ignored due to path filters: 1
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing: 23
  • .changeset/human-actor-and-ignored-trace.md
  • docs/debugging.md
  • docs/decisions.md
  • docs/environment-and-policy.md
  • docs/evals.md
  • docs/observability.md
  • docs/verify.md
  • examples/README.md
  • examples/mcp-workflow/index.test.ts
  • examples/mcp-workflow/index.ts
  • examples/mcp-workflow/metadata.json
  • package.json
  • src/decide-next.ts
  • src/describe-machine.ts
  • src/events.ts
  • src/fold-traces.test.ts
  • src/fold-traces.ts
  • src/human-actor.test.ts
  • src/interaction.ts
  • src/run-agent.test.ts
  • src/run-agent.ts
  • src/validate/verify-config.test.ts
  • src/verify.ts

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 0 remain after this review.

📝 Walkthrough

Transitions tagged actor: "human" are excluded from agent decision options and identified by machine descriptions and linting. Host-event traces now mark ignored transitions, and foldTraces counts them per state. The changes also add an MCP bug-fix workflow example.

Changes: Human-tagged transitions

Layer / File(s) Summary
Human event contract and decision filtering
src/interaction.ts, src/events.ts, src/decide-next.ts, src/run-agent.ts, src/human-actor.test.ts, docs/decisions.md, docs/environment-and-policy.md
The metadata schema accepts actor: "human". Agent event selection excludes tagged event types and reports excluded candidates in withheld.
Machine descriptions and decision linting
src/describe-machine.ts, src/verify.ts, src/human-actor.test.ts, src/validate/verify-config.test.ts, docs/debugging.md, docs/verify.md
Machine descriptions label human-tagged transitions. Linting reports literal decision event lists that include events handled by human-tagged transitions.
MCP bug-fix workflow example
examples/mcp-workflow/*, examples/README.md, package.json
Adds an MCP workflow with run status, agent-sent events, and separate human event delivery. Tests cover tool calls and workflow outcomes.

Ignored transition trace counts

Layer / File(s) Summary
Ignored transition markers and per-state statistics
src/run-agent.ts, src/fold-traces.ts, src/run-agent.test.ts, src/fold-traces.test.ts, docs/evals.md, docs/observability.md, .changeset/human-actor-and-ignored-trace.md
Ignored host-event transitions carry ignored: true in traces. foldTraces counts those events for the active state. The tests and documentation describe the marker and count.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Agent
  participant runAgent
  participant getHumanEventTypes
  participant Snapshot
  runAgent->>getHumanEventTypes: identify human-tagged event types
  getHumanEventTypes->>Snapshot: inspect available transitions
  getHumanEventTypes-->>runAgent: return human event types
  runAgent->>Agent: send decision options without human events
  runAgent->>runAgent: record excluded candidates in withheld
Loading

Possibly related PRs

  • statelyai/agent#173: Changes the decision-request and event-selection contract that this PR extends for human-tagged events.
Merge Risk: ⚪ Minimal · up to 17642

No confirmed issue currently blocks merging; complete the normal project checks before release.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 64.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 25 functions across 13 files. (10 skipped… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names three changes: human-sent transitions, ignored events in traces, and the MCP workflow example. All three appear in the changeset. The title is long (70 characters, 9 words) and lists t…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names three changes: human-sent transitions, ignored events in traces, and the MCP workflow example. All three appear in the changeset. The title is long (70 characters, 9 words) and lists three separate features instead of one main change. It is still accurate, and a reviewer scanning history can tell what the PR does.

Full details: Docstring Coverage

Explanation

Docstring coverage is 64.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 25 functions across 13 files. (10 skipped: 10 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 4 potential issues.

Devin Review

Comment thread examples/mcp-workflow/index.ts Outdated
Comment thread src/verify.ts Outdated
Comment thread src/events.ts Outdated
Comment thread examples/mcp-workflow/index.ts
…vents, step-path decisions skip human events
@davidkpiano
davidkpiano merged commit 86e9d30 into next Oct 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant