Skip to content

feat(examples): agent-wakeups hook — wake the agent only for its own checks, notifications and the person - #352

Draft
sarah-chitty-1 wants to merge 30 commits into
sarahchitty/agent-wakeupsfrom
sarahchitty/agent-wakeups-hook
Draft

sarah-chitty-1 wants to merge 30 commits into
sarahchitty/agent-wakeupsfrom
sarahchitty/agent-wakeups-hook

Conversation

@sarah-chitty-1

@sarah-chitty-1 sarah-chitty-1 commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #351 (wakeups); #265 is merged. The diff is only this PR's hook. It'll retarget to main once #351 merges.

What and why

This is a post-turn hook for prompt_agent that runs one conversation over simulated days. It wakes the agent only for what would really reach it:

  1. a check the agent scheduled for itself (schedule_check, feat(gateway): wakeups — checks an agent schedules for itself, and notify_agent #351)
  2. a notification a trigger sent it (notify_agent, feat(gateway): wakeups — checks an agent schedules for itself, and notify_agent #351)
  3. a message the person writes at a set time (person_messages)

Everything else the world does happens silently while the agent is away. The agent finds it only if it goes and looks. Each wake line names its time and source, so nothing poses as the person:

[Tue Mar 3, 6:00 PM · your scheduled check] look for new rugs
[Wed Mar 4, 8:00 AM · Messages] Jamie: did the rug thing get sorted?

How each turn goes:

  • Settle the triggers.
  • Let a firing agent trigger answer first.
  • Otherwise, take what's due. If nothing is due, jump the frozen clock to the earliest of: the next world event (next_event_at), the next scheduled check, or the next person message. Repeat.
  • The run ends when nothing can wake the agent, at end_time, or after max_jumps_per_turn.

It's shipped as an example bundle (examples/agent-wakeups). The only framework change: post_turn_hooks._INVOKABLE now includes the wakeups extension, so hooks can call take and state.

How it was tested

  • tst/unit/task_step/test_agent_wakeups_hook.py: 8 passed, run against an in-process gateway's real clock, triggers and wakeups. It covers:
    • a silent world event
    • a notify_agent wake
    • person messages arriving once each
    • items due together going into one message
    • end_time
    • a trigger answering before any jump
    • repeating checks
    • param validation
  • tst/integration/task_step/test_agent_wakeups_hook_local.py (-m int_test_slow) passed in 114 s on a real local Docker gateway. The world event landed silently, and only the agent's own check and a declared notification woke it, with exactly the expected lines.
  • Real-model runs: Curbfinch furnish-week on the PSI laptop self-host, three runs on Opus 5.5 at d4847a8. The clock was frozen and jumped by this hook. The rug listing was silent, and seller replies were sent with notify_agent ("Curbfinch message").
    • On turn 1 of every run, the agent called schedule_check for every 2–4 h, overnight too. It rescheduled as things changed and cancelled its checks when it was done.
    • Seller notifications woke it only on evenings with real replies. Quiet evenings caused no wake.
    • All 3 runs found and bought the silently listed $360 rug at the first check after it appeared, for $320, $310 and $300.
    • Runs 2 and 3 ended with nothing_can_wake_the_agent on Friday evening. Run 1 hit a 15-turn cap.
    • Cost per run: $5.69, $6.26 and $8.02.
    • The site read the jumped gateway clock on every call, with no re-sync needed.
  • Stress test: 6 runs on Opus, GPT and Gemini, 2 each, over a 10-day Curbfinch rug hunt with a decoy listing and a day-3 change of plan.
    • 43 schedule_check calls, none refused. They used relative and absolute times, until both as an instant and as a duration, and hourly repeats. Cancelling and rescheduling worked.
    • Every alert arrived.
    • The scripted person message arrived once, at its time.
    • The step and jump limit was never hit.
    • Every run that wasn't stopped by its cost cap ended with the right reason.
    • No wakeups bugs were found. Cost was driven by the harness resending the whole 10-day transcript on every call, not by the wakeups code.

@earakely-scale

🤖 Generated with Claude Code

sarah-chitty-1 and others added 19 commits October 19, 2026 16:37
…to the next on simulated time

prompt_agent's opt-in between_turns: next_env_event makes a multi-turn run
event-driven, for long-running and background-agent evals whose events are
hours or days apart. After each agent turn it reads the envs' trigger state,
brings the armed env clocks up to the earliest pending time-trigger mark,
waits for the triggers due there to fire, fail or be cancelled (with a
timeout), and wakes the agent with a neutral message or the user simulator's
agent-trigger decide step. It stops when no time trigger is pending, at
end_time, or at max_conversation_turns. The user side is never asked.

The gateway clock gains PUT /clock/advance (clock/v1 "advance"): a forward
jump on the current arm that keeps t0, rate and generation, so relative and
recurring time-trigger marks keep their meaning and delays count the jumped
time. set_time is a re-arm that re-anchors them; against a gateway without
advance, prompt_agent re-arms at the env's own rate and refuses a run whose
pending time triggers are relative or recurring.

With between_turns set, or record_env_clock, every conversation message
prompt_agent records carries env_clock, each env's virtual time when it was
sent or received, so a grader can place the agent's messages on the same axis
as the trigger firing log and the gateway trajectory.

Both options are off by default; unset, prompt_agent, the conversation store
and the clock behave as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two envs can each have a trigger with the same id due at one mark; keyed by id
alone, the second was skipped and the wait ran to its timeout. Outcomes are now
keyed by (env id, trigger id) and recorded as {env_id: {trigger_id: outcome}}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dvance lands is not an error

With a running clock the mark can pass between the clock read and the advance,
which the gateway then refuses as backwards. On that 400 the clock is read again,
and if it has reached the mark the move is recorded as "passed"; any other
refusal still fails the step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…next mark in the schedule

A recurring trigger whose fire outlived the wait (an action delay on a frozen
clock, say) stays firing with its next mark already stamped; only armed ones
were scheduled, so the run could end with "no_pending_time_triggers" despite
future marks. Firing and queued triggers with a next mark are now scheduled too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… with carries the env clock too

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ect with 400, not a server error

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…gain

A recurring trigger whose fire outlives its interval on a frozen clock stays
firing with the same next mark, so every later turn chose that mark again,
waited out the timeout and woke the agent without moving time. Marks that timed
out are now skipped, so the run moves on to the next event or end_time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rring mark, as every other jump does

Against a gateway without clock advance, stopping at end_time re-armed the clock
unchecked, re-anchoring a pending relative or recurring time trigger past
end_time for the steps after it. That jump now runs the same check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… stored message carries the env clock

Hooks are Python FileArtifacts defining `async def post_turn(turn, params)`, run in
order after every agent turn; the first to return a message or an end decides, else
the user side is asked as today, which moved into _UserSide unchanged.
test_prompt_agent_user_side.py pins that path and passes unchanged.

Agent triggers/v1 decide takes record_unfired, so user_triggers() can ask again in
the same turn without resetting once-only triggers. The verifier run context reads
env_clock as one EnvClock per armed env.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t_event_at and resumes_at; /step tags a hook's calls

PUT /clock/advance runs the trigger time check at once and returns `due`, waking
actions that wait out a delay. /triggers/state reports next_event_at, the earliest
time the engine acts on its own, and each trigger's resumes_at (triggers/v1 feature
next_event_at). /step takes source post_turn_hook with hook_artifact_id (step/v1
feature source).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…om env event to env event

The hook waits for what the agent's turn set off, lets a firing agent trigger answer,
advances each clock to the earliest next_event_at, waits for it to settle, then asks
the agent triggers again or wakes the agent. It ends when nothing is pending or past
end_time, and on a gateway without advance. It ships as a bundle with an inbox env
and a week-long task; the slow test runs it against the echo agent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both sides add a fourth shipped bundle, mail-mcp and next-env-event: the entry points
and the shipped-bundle lists keep both, and the installer journey expects 5 bundles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…so a change to it fails the test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e next opening

_await_hours reckoned the opening again only when the clock's generation changed,
and advance keeps it, so a jump past the close left a stale opening behind and the
executor ran outside its hours. It now reckons the opening on every pass, and
clock_advanced counts a wait as ended only while its window is open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ps-hook

# Conflicts:
#	src/agent_env/env/gateway/constants.py
#	tst/unit/bundle/installed_test.py
#	tst/unit/bundle/shipped_bundles_test.py
#	tst/unit/test_clean_install_wheel.py
…checks, notifications and the person

A prompt_agent post-turn hook (example bundle agent-wakeups) that runs a conversation over simulated days:
after each turn it jumps the clock, silently, to the next thing that happens — a world event, a check the agent
scheduled (wakeups/v1), or a person's message at a set time — and wakes the agent only when one of its own checks,
a notify_agent notification or a person's message has come due. World events with no channel wake no one, so the
agent isn't told when to look. Each wake is one message, a line per thing with its time and source
("[Tue Mar 3, 6:00 PM · your scheduled check] …", "[Wed Mar 4, 8:00 AM · Messages] …"), so nothing poses as the
person; the hook's record names what each wake was for.

post_turn_hooks: a hook may also invoke wakeups/v1 (state, take).

Stacked on #265 (post-turn hooks, clock advance, next_event_at) and on the wakeups/v1 gateway change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sarah-chitty-1 and others added 10 commits October 27, 2026 10:01
Resolves the bundle-list conflicts in three tests: main added the drivers bundle, this branch next-env-event; keep both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ler expects six bundles

- /triggers/state shows each trigger's `evaluating`, and the next-env-event hook waits for a state check the agent's
  turn set off before it moves the clock
- test_container_journey: next-env-event makes six shipped bundles

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	tst/unit/bundle/installed_test.py
#	tst/unit/bundle/shipped_bundles_test.py
#	tst/unit/test_clean_install_wheel.py
… set off before it moves the clock

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aclasses and postponed annotations work

exec_module doesn't put the module in sys.modules, and dataclasses looks it up there. The name gets a random suffix
(concurrent runs may share a step id) and close removes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	src/agent_env/task/episode/rules.py
…ps-hook

# Conflicts:
#	pyproject.toml
#	src/agent_env/task_step/task_steps/post_turn_hooks.py
#	tst/installer/test_container_journey.py
#	tst/unit/bundle/installed_test.py
#	tst/unit/bundle/shipped_bundles_test.py
#	tst/unit/test_clean_install_wheel.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant