[sarah-chitty-1]

What

Some evals run an agent through days of simulated time and grade how it reacts to each event. This PR lets one conversation go from event to event: each turn takes real seconds, and the time between events is skipped. It follows the design from review. The gateway owns the clock jump and knows when its triggers next act. The event-to-event logic is a hook script that a task references.

• prompt_agent.post_turn_hooks (opt-in) is a list of Python FileArtifacts, each defining async def post_turn(turn, params).
  • They run in order after every agent turn, the last one included.
  • The first to return {"message": ...} or {"end": reason} decides the turn. If none does, the user side is asked as today.
  • turn carries the turn's number, whether it is the last, the agent's reply, the seed, a read-only copy of the metadata with secret keys stripped, and the envs. Each env has supports, invoke (clock/v1 and triggers/v1) and call_tool.
  • turn also offers ask_user(), user_triggers(), record() and heartbeat().
• PUT /clock/advance jumps the clock forward on its current arm and runs the trigger time check at once. It returns due: the triggers the jump reached, plus those whose wait it ended. Waiting actions wake straight away.
• /triggers/state gains next_event_at, the earliest simulated time the engine acts on its own: a time trigger's mark, a delayed action resuming, or an executor's hours opening. Each trigger also gains resumes_at. Both sit behind the triggers/v1 feature next_event_at.
• /step takes source: "post_turn_hook" with a hook_artifact_id, so the trajectory records a hook's tool calls as the hook's (step/v1 feature source).
• The agent triggers/v1 decide (protocol package) takes an optional record_unfired (default true). With false, a decision where nothing fires changes no state, so user_triggers() can be asked again later in the same turn.
• env_clock is on every stored conversation message, hook messages included. It holds one entry per armed env clock (env_id, env_step_id, virtual_time) and is absent when no clock is armed.
• The next-env-event example bundle (src/agent_env/examples/next-env-event/) holds the hook, a small inbox env, and a week-long task that uses them. After each turn the hook:
  1. waits for what the agent's turn set off to finish;
  2. lets a firing agent trigger answer first, with no jump;
  3. finds the earliest next_event_at, and ends at end_time or when nothing is pending;
  4. advances every clock that is behind and waits for it all to settle;
  5. asks the agent triggers again, or wakes the agent with a nudge.

between_turns, record_env_clock and the set_time fallback are gone.

Worth a look

• With no hooks, the step behaves as before. Today's user-side code moved into _UserSide unchanged.
  • test_prompt_agent_user_side.py, which lands on main first in test(prompt_agent): pin down a multi-turn conversation's user side as it runs today #331, records every request and store write for each way a turn can end. It passes here unchanged.
  • The one addition: for an env whose card offers clock/v1 state, prompt_agent now reads /clock/state when it records a message (two GETs a turn, a 5 s timeout each). A failed read leaves that env out of the stamp. A test pins that this, and the env_clock field when a clock is armed, are the only differences.
  • The user message is stamped just before it is sent, ahead of any file staging, so at a high clock rate its stamp can lead the stored message slightly.
• The ask-once rule needed a protocol change. The shipped agent trigger engine reads a second decide for the same turn as a restart: it clears what fired, so once-only triggers fire again, and it logs a reset.
  • user_triggers() therefore sends record_unfired: false. A call where nothing fires leaves no trace and can be repeated. Once the user side answers, later calls that turn return the same answer, and a hook that asked must return it, or the step fails.
  • A user agent whose card doesn't declare the field is refused when a hook calls user_triggers(), naming the fix, instead of silently re-firing its once-only triggers.
  • ask_user() and the no-hook path send today's decide unchanged.
  • The shipped agents' card-contract tests gain the field. Their images advertise it once rebuilt on this protocol.
• The human peer URL. Without hooks it is still resolved before the first turn, so a missing one still fails before any agent spend. With hooks it is resolved only when a hook asks the user.
• The verifier run context (feat(verifiers): give verifiers the run's conversation and trigger firing logs #241). Message.env_clock now reads the list shape, as a tuple of EnvClock(env_id, env_step_id, virtual_time), and is still None for a message stored without a clock. Nothing on main writes env_clock yet. plugin-api: no break.
• next_event_at:
  • It reads marks the engine hasn't stamped yet, such as right after a register or a re-arm, without stamping them. A stochastic every is read from a copy of its RNG.
  • A mark past until, an exhausted count, an after trigger whose anchor hasn't fired, and the next mark of a trigger still mid-fire don't count. A waiting fire's resumes_at stands in for that last one.
• A waiting action now sleeps in a race between its recheck and an advance (TriggerEngine._nap). So an advance wakes it at once, instead of up to 5 s later.
• The hook waits for more than due. It waits until no trigger on the env is still firing, bar one waiting for a later time, both before reading the next event and after the jump. That way a reply the agent's own last turn set off lands before the clock moves.
  • When an event is already reached but not yet picked up, it runs the time check with a zero-length advance, instead of waiting for the 1 s poller.
  • It is made for a frozen clock. On a running clock, a trigger that recurs faster than a turn makes each settle wait out its 300 s timeout.
• Trust. A hook runs in the control-plane process with the trust of an env_outcome_verifier script. The /step source tag is self-declared, with the same trust as the AgentEnv-Role header.
• The last turn. Hooks run there too, and a returned message is ignored, so a hook should check turn.last before asking the user anything.
• A fifth shipped bundle, beside main's new mail-mcp. The bundle-count tests and the installer journey's expected row (now "5 bundles") are updated. The example's task needs Docker and a model endpoint. CI checks that the bundle parses and plans, and the slow tier runs its hook against the echo agent. A bundle needs a task to pass the shipped-bundle check, which is why the inbox env and the task are there.
• Older gateways. Envs need a gateway image built from this branch for advance, next_event_at and the step tag. On an older one the hook logs a warning naming the env and ends without jumping, and call_tool refuses, naming the env.

Tests

• Unit, new or rewritten:
  • tst/unit/env/gateway/next_event_test.py (20): next_event_at before and after evaluation, recurring and stochastic marks, re-arm, until/count, after, a mid-fire recurrence, and fired or unarmed; resumes_at for a time-trigger delay, an action delay and executor hours; what an advance makes due, and waking a delayed action at once; the /step tag and its refusals; and the card features.
  • tst/unit/task_step/test_prompt_agent_post_turn_hooks.py (38): the field, its validation and entity refs, and a bundle carrying the hook artifact; loading; each decision, including order, the user-side fallback, the lazy human peer, bad returns and the last turn; the ask-once rule, including the real agent trigger engine behind /decide (a once-only trigger fires once however often a hook asks), refusing a user agent without record_unfired, and two concurrent asks asking once; what turn holds; call_tool through /step and its refusals; and env_clock, including that without hooks a clock-capable env only adds clock reads.
  • tst/unit/task_step/test_next_env_event_hook.py (17): the hook turn by turn, against an in-process gateway with a real Clock and TriggerEngine. It includes both cases from review: the PT2H action lands at Tue 11:00 with a wake of its own, even with no later event, and an agent trigger's reply goes first with no jump. It also covers a recurrence whose delayed action outlasts its interval, a fire the agent's turn set off landing before the jump, end_time as an instant and as a duration, a custom nudge, no armed clock, an older gateway, and bad params.
  • packages/agentenv-protocol/tests/test_a2a_agent.py: a decision where nothing fires can leave no trace.
  • tst/unit/task_step/test_prompt_agent_user_side.py (17), unchanged from test(prompt_agent): pin down a multi-turn conversation's user side as it runs today #331.
  • clock_advance_test.py, test_run_context.py, shipped_bundles_test.py, installed_test.py, test_clean_install_wheel.py, the installer journey and three agents' card-contract tests were updated.
• Integration, slow tier, local defaults: tst/integration/task_step/test_next_env_event_hook_local.py (2) runs on a gateway built from this branch, with the items env and the echo agent.
  • Multi-day: two events days apart wake the agent once each, and every message carries the env clock.
  • Delayed action: a PT2H action lands at its mark plus two hours, with its own wake, before the next event.
  • Result: 2 passed (178 s).
• Ran:
  • ruff check: clean. check_plugin_api.py --base origin/main: no break. uv lock --check: ok.
  • Full unit suite (tst/unit, packages/agentenv-protocol/tests): 10535 passed, 2 failed. Both failures are in test_mcp_cli_builder.py: the scripts it generates run this machine's system python3, which has no click. They fail on main too.
  • Existing integration tests, 22 passed on the local provider:
    • test_register_env_triggers_local.py (5: delays and hours, deadlines, named executors, personas, the start-of-run await).
    • test_prompt_agent_live_trajectory_local.py (1) and test_prompt_agent_file_parts_local.py (3).
    • gateway_test.py (13): triggers, barrier, re-arm after a failed mirror, fail-loud registration, queued-not-settled, website trigger, clock, clock rate, time trigger, autonomous time trigger, sync_env_clock, single MCP server, and changes-name-the-role.
    • Their [modal] and [modal_vm] variants ran because this machine has Modal credentials, then failed on Modal itself: the environment is over its spend limit, and Modal can't pull images from a local registry. test_gateway_triggers_nl_executor runs only on modal_vm, so it didn't run.
  • Not run: the installer tier, and the external-source clock-consumer tests (AGENT_ENV_TEST_MCP_SERVERS_DIR). The read contract those check (GET /clock/time, /clock/state) is unchanged.

Docs (for the docs site)

• prompt_agent post_turn_hooks: the hook contract, the turn API, how a turn is decided, the ask-once rule, the trust model, and that in a hook run the LLM user sim sees only the replies sent through ask_user().
• clock/v1 advance and its due; triggers/v1 next_event_at and resumes_at; step/v1 source; agent triggers/v1 decide's record_unfired.
• The env_clock field on conversation messages.
• The next-env-event example bundle and its params.

🤖 Generated with Claude Code

Confidence Score: 5/5

The latest changes appear safe to merge.

Summary

This PR adds post-turn hooks, clock jumps, and an example that moves a conversation between simulated events.

• Tasks can run hooks after turns to choose what happens next.
• The gateway can jump simulated time to the next trigger event.
• The shipped inbox example wakes one conversation at scheduled env events.
• Conversation messages record the simulated time for each armed env clock.

_{Reviews (13) · Last reviewed commit: "fix(post-turn-hooks): register a hook's ..." · Reviewed by Greptile}

[earakely-scale]

@sarah-chitty-1 thanks for this, the use case is a good one. We went through the PR in detail and would like to reshape it. Keep the clock primitive, and move the event-to-event logic out of prompt_agent into a hook script you own. Below are the target design and the two problems we hit in the current version.

Two problems in the current version

These were reproduced against a real Clock + TriggerEngine, driving the PR's own between-turns logic.

1. Delayed actions don't get a wake of their own. The test was a frozen clock and a trigger at Tue 09:00 whose action has delay: PT2H. The wait timed out, and the action ran at the next event, Thu 09:00, instead of Tue 11:00. With no later event and no end_time, the run stopped and the action never ran. The cause: /triggers/state shows a waiting action as firing with no time attached, so the next-event search can't see when it's due. Executor working hours should behave the same way (from reading _await_hours).
2. wake: "decide" runs after the jump. decide is sent the agent's last message, so a user-sim reply to it lands at the next event's simulated time. An agent trigger that reacts to the final turn never runs, because the run stops before decide.

Target design

1. PUT /clock/advance: keep it as you wrote it, plus one addition

The semantics stay as they are: a forward jump on the current arm, with t0, rate and generation unchanged, the jump counted in elapsed(), and the 400s. The addition: right after the jump, the route runs the engine's time check, the same synchronous call the 1 s poller makes, and returns the triggers that came due:

PUT /clock/advance {"virtual_time": "2026-03-03T09:30:00Z"}
→ 200 {..., "virtual_time": "2026-03-03T09:30:00Z", "due": ["school-email"]}

Detection is then exact on running clocks too, and callers know what to wait for without polling. Optionally, it can also wake actions that are waiting out a delay, which otherwise notice the jump on their next recheck, up to 5 s later.

2. prompt_agent.post_turn_hooks, replacing between_turns

A list of hook scripts. Each is a Python FileArtifact, loaded the way env_outcome_verifier loads its script, that defines one function:

"post_turn_hooks": [
  {"file_artifact_id": "next-env-event", "file_artifact_version": 3, "params": {"end_time": "P4D"}}
]

async def post_turn(turn, params) -> dict | None:
    return None                          # no decision: the next hook runs
    return {"message": "..."}            # the next user message (text or parts)
    return {"end": "no_pending_events"}  # stop; may carry a closing "message"

• Who decides. Hooks run in list order after each agent turn, and the first one to return a message or an end decides. If none decides, the user side is asked exactly as today. With no hooks, the step is unchanged.
• Hook artifacts are declared as entity refs, so bundles and exports carry them. Each hook is loaded once per step run.
• What turn carries:

turn.number, turn.last          # last: after the final turn; a returned message is ignored
turn.agent_reply, turn.agent_parts
turn.seed, turn.metadata        # a read-only copy of the run's metadata, secret keys stripped
for env in turn.envs:
    env.env_id, env.env_step_id, env.instance_id
    env.supports(extension, method)
    await env.invoke(extension, method, params)   # the published clock/v1 and triggers/v1 calls
    await env.call_tool(tool, args, role)         # gateway envs; through /step (see below)
await turn.ask_user()           # today's path: agent triggers, then the LLM user sim or a human
await turn.user_triggers()      # agent triggers only; None if none fire
turn.record(entry)              # -> context.metadata["post_turn_hooks"][step_id], tagged with the hook's artifact id
turn.heartbeat()

• call_tool goes through the gateway's existing /step route, so role rules, trajectory logging, action triggers watching that role, and barriers all behave as for any caller. The role is required, and any role is allowed. Each call is tagged in the trajectory as source: post_turn_hook plus the hook's artifact id, the way trigger actions are tagged source: trigger_engine, so graders can tell hook calls from the agent's. That needs a small /step change.
• Rules:
  • Hooks must be async: steps and runs share one event loop.
  • Hooks run after every completed agent turn, including the last one.
  • A hook has the same trust model as env_outcome_verifier: task-referenced code running in the control-plane process. The docs should say so.
  • The human peer URL is looked up only when ask_user() needs it.
• Left out on purpose: the task context object, the conversation history, and the agent's address. A hook speaks to the agent only by returning a message, so core records everything the agent receives.

3. Core additions the hooks need

• env_clock on every stored conversation message, hook messages included. It's always on, with no flag, and has one entry for each env whose clock is armed. The field is absent when no clock is armed:

  "env_clock": [{"env_id": "items", "env_step_id": "deploy_items", "virtual_time": "2026-03-03T09:30:00Z"}]

  env_step_id tells two deployments of one env apart, the way deploy_agent does.
• next_event_at in /triggers/state, behind a triggers/v1 feature flag. It's the earliest simulated time the engine will act on its own: armed marks, when a waiting action resumes (also per trigger as resumes_at), and executor-hour openings. This fixes problem 1 at the source. The engine already knows all three times.

4. Your next-env-event logic as a hook script

It ships as an example bundle under src/agent_env/examples/, which keeps your slow-tier integration test, plus a delayed-action case. Each turn it does five things:

1. Reply first: await turn.user_triggers(). If a trigger fires, return its reply, at the current time, with no jump. This fixes problem 2.
2. Find the earliest next_event_at across envs with an armed clock. End on end_time (an instant or a duration from t0), or when nothing is pending.
3. advance every clock that's behind it.
4. Wait until the triggers in due settle, or are waiting for a later resumes_at.
5. await turn.user_triggers() again, or send the nudge (only {virtual_time} as a placeholder).

On a gateway without clock advance or next_event_at, it logs a warning naming the env (rebuild its gateway image) and ends without jumping. There is no set_time fallback: a re-arm moves relative and recurring triggers, and an older gateway can't time delayed actions anyway.

Zero regression for the user sim

With no hooks, the user side must behave exactly as today:

• the same calls in the same order, with the same payloads, context ids and timeouts;
• the same env_trigger_snapshots, agent_trigger_firings, usersim_turn_outputs and firing-log readback;
• the same end conditions.

The LLM user sim's done message stays unrecorded, as today. The only intended difference is the additive env_clock field.

Today's tests barely cover this loop, so a characterization test of the current user-side behaviour should land on main first. It would cover the LLM sim's reply / done / malformed JSON / extra fields / timeout / non-completed / empty parts; agent triggers fired / done / none fired; the human peer; and external close, agent failure and the turn limit. This PR should then pass it unchanged.

Scope and review order

One PR, reviewed in this order:

1. the prompt_agent seam;
2. the gateway pieces (advance + due, next_event_at / resumes_at, the /step source tag);
3. your hook script and its example bundle.

Still open, hook runs only

• Ask-once rule. Your hook calls user_triggers() before and after the jump. Proposed: a call where nothing fires leaves no trace and can be repeated. Once the user side answers (a message or an end), later calls that turn return the same answer, and a hook that asked must return it.
• What the user sim sees. In a hook run, the LLM user sim only receives the agent replies sent through ask_user(), so turns a hook answered are missing from its history. We'll document that for now.

Happy to talk any of this through.

[sarah-chitty-1]

@earakely-scale thanks, this was a much better shape. I reworked the PR to it, and the description is updated. In your review order:

1. The prompt_agent seam. post_turn_hooks works as you specified: list order, the first to decide wins, the turn API, entity refs, each hook loaded once per step run, and async only. Today's user-side code moved into _UserSide unchanged. The characterization test, landing on main first in test(prompt_agent): pin down a multi-turn conversation's user side as it runs today #331, passes here unchanged. The human peer URL is still resolved up front when there are no hooks, so a missing one still fails before any agent spend. With hooks it is resolved only when a hook asks.
2. The gateway pieces.
   • advance returns due and wakes waiting actions at once.
   • next_event_at and resumes_at sit behind the triggers/v1 feature next_event_at. A trigger still mid-fire doesn't report its next mark, since it can't fire again until that fire ends.
   • /step takes source: "post_turn_hook" plus hook_artifact_id. It is declared as the step/v1 feature source, and call_tool refuses a gateway that lacks it.
3. The hook. It lives in src/agent_env/examples/next-env-event/ and has no set_time fallback. Your two repros are now unit tests against a real Clock + TriggerEngine: the PT2H action lands at Tue 11:00 with a wake of its own, also with no later event, and an agent trigger's reply goes first with no jump. One change to step 4: the hook waits until nothing on the env is still firing, not just what's in due, and it does that before reading the next event too. That way a reply the agent's last turn set off lands before the clock jumps. The slow test keeps the multi-day case and adds the delayed-action one.

On your open questions:

• Ask-once: agreed, but it needed one change underneath. The shipped agent trigger engine reads a second decide for the same turn as a restart: it clears what fired, so once-only triggers fire again. So triggers/v1 decide in the protocol package now takes an optional record_unfired (default true), and user_triggers() sends false.
  • A call where nothing fires leaves no trace and can be repeated.
  • Once the user side answers, later calls that turn return the same answer, and a hook that asked must return it, or the step fails.
  • A user agent whose card lacks the field is refused when a hook calls user_triggers(), rather than re-firing its once-only triggers.
  • ask_user() and runs without hooks send decide as before.
• The user sim's history: documented on ask_user(), and listed in the PR's docs section.

Two other things that go beyond the spec are in "Worth a look":

• The verifier run context (feat(verifiers): give verifiers the run's conversation and trigger firing logs #241) now reads the list-shaped env_clock.
• The example bundle has a small inbox env and a demo task, because a bundle needs a task to pass the shipped-bundle check.