Skip to content

test(prompt_agent): pin down a multi-turn conversation's user side as it runs today - #331

Merged
sarah-chitty-1 merged 2 commits into
mainfrom
sarahchitty/prompt-agent-user-side-characterization
Oct 26, 2026
Merged

sarah-chitty-1 merged 2 commits into
mainfrom
sarahchitty/prompt-agent-user-side-characterization

Conversation

@sarah-chitty-1

@sarah-chitty-1 sarah-chitty-1 commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

What

A characterization test of prompt_agent's user side, so the rework in #265 can show it leaves that side unchanged. Each test records every request a conversation makes, in order, and what it stores. The requests cover the agent, the user sim or a human peer, agent-trigger decide, env trigger state, and the firing-log readback. The records cover the stored conversation and the env_trigger_snapshots, agent_trigger_firings, usersim_turn_outputs and agent_trigger_state metadata.

The cases:

  • The LLM user sim: a reply, done (its message stays unrecorded), malformed JSON, a reply missing done, extra fields, a timeout (its task is cancelled), a task that fails, is canceled or is rejected, and a reply with no parts.
  • Agent triggers: one fires and speaks for the user, one ends the conversation with a closing message, or none fires and the user sim answers.
  • A human peer: named by user_a2a_url, and the configured default.
  • How a run ends: a conversation closed from outside, an agent failure, and the turn limit.

Worth a look

  • Test only: no code changes.
  • Peers are answered through an httpx MockTransport and the conversation goes to the real local store, so the test pins requests and stored records, not function calls. A refactor that keeps the behaviour passes it unchanged. Ids are normalized: the conversation's is <ctx>, and the others are <idN> in order of appearance.
  • The env in the agent-trigger tests offers only triggers/v1, so a change that reads an env's clock wouldn't show up in these request logs.

Tests

  • The new file: 17 passed.
  • Full unit suite (tst/unit, packages/agentenv-protocol/tests): 10442 passed, 2 failed. Both failures are in test_mcp_cli_builder.py, whose generated scripts run this machine's system python3, which has no click. They're unrelated to this change.
  • ruff check: clean. uv lock --check: ok.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

This test-only PR appears safe to merge.

Summary

Adds tests that pin down requests and stored records for prompt_agent conversations.

  • Prompt-agent tests now pin down simulator turns and run endings.
  • Prompt-agent tests record when agent triggers speak for the user.
  • Prompt-agent tests cover replies from an explicitly chosen or default human peer.

Reviews (2) · Last reviewed commit: "review: the user-side test pins the user..." · Reviewed by Greptile

… it runs today

Each test records every request a conversation makes, in order, and what it
stores, for each way a turn can end: the LLM user sim's reply, done, malformed
output, extra fields, timeout, a task that doesn't complete and empty parts; agent
triggers fired, done and none; a human peer; and external close, agent failure and
the turn limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread tst/unit/task_step/test_prompt_agent_user_side.py Outdated
…so a change to it fails the test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sarah-chitty-1
sarah-chitty-1 merged commit 3b1c71e into main Oct 26, 2026
14 checks passed
@sarah-chitty-1
sarah-chitty-1 deleted the sarahchitty/prompt-agent-user-side-characterization branch October 26, 2026 22:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants