[abnegate]

Summary

A full verify run on a heavily loaded host (load average ~80, several workspaces compiling) failed two tests the PR under test never touched. One failure was a real bug; the other was a test that relied on the machine being fast.

zone_comfy train::tests::a_graph_that_never_finishes_times_out: production bug

• wait_prompt set each /history poll's request timeout to whatever was left of the training deadline (min(remaining, 5s)).
• If ComfyUI was still answering when the deadline arrived, reqwest's own timeout fired first. The job then failed with error sending request for url (…/history/<id>) instead of training timed out.
• Under load the deadline usually lands during a poll, which is why the test got the reqwest error. The 30 s it took was cancel_and_wait polling a mock graph that never ends.
• Fix: each poll keeps the fixed 5 s REQUEST_TIMEOUT, and the whole poll runs under tokio::time::timeout_at(deadline, …). If the deadline comes first, the job fails with training timed out. If a request fails before the deadline, that failure is reported. cancel_and_wait limits its own polls to its deadline the same way.
• The timeout tests now mount a mock ComfyUI that marks the prompt terminal once it is cancelled. They assert that the cancellation was confirmed (cleanup == true), and they no longer spend 30 s polling.

zone_server chat_context_store_tests::catalog_pages_all_references_without_exposing_result_bodies_or_other_chats: test-only

• The test takes a 30 s lease and never renews it. It then makes several hundred store round trips, because it pages the catalog 7 characters at a time.
• It takes about 1 s on its own. With every test binary sharing one Postgres under load it took 39 s, and the final delete_message failed with LeaseLost.
• Production never holds a lease without renewing it: services/chat/session.rs pairs every acquire with keep_alive. The store has no bug.
• Fix: the test now holds a keep_alive guard like a generation does and asserts the lease was never lost. Its assertions about paging, redaction, chat and workspace scoping, and snapshot invalidation are unchanged.

refactor(chat): box the route a chat turn carries

• Clippy 1.98 fails zone_server on main with large_enum_variant on ws::chat::Routing.
• This is a cherry-pick of the commit already on migrate/sanitize, migrate/rmcp and migrate/notify-email, with an identical diff, so whichever PR lands first leaves the others a no-op.

Behaviour differences

ComfyUI training (zone_comfy, regression test a_comfyui_too_slow_to_answer_still_times_out_at_the_deadline)

• A training run whose deadline arrives while ComfyUI is answering a status poll now fails with training failed: training timed out. It used to fail with training failed: error sending request for url (…). Either way the prompt is cancelled and the run is cleaned up.
• A status poll that starts more than 5 s before the deadline is now limited only by the 5 s request timeout. One that starts later is cut off by the deadline itself. Before, the poll's own request timeout was shortened to the time left. The deadline still ends training at the same moment.
• Known edge, unchanged from before: a poll that starts just over 5 s before the deadline and never gets an answer still fails with reqwest's own timeout error (error sending request for url (…)) a few milliseconds before the deadline.

No other behaviour changes. The chat change is test-only, and the Routing change only boxes a field.

Verification

• Fail-before: with the production change reverted and only the new tests kept, a_comfyui_too_slow_to_answer_still_times_out_at_the_deadline fails with training failed: error sending request for url (http://127.0.0.1:…/history/…), the same error as the original CI failure. With the fix it passes.
• cargo nextest run -p zone_comfy --no-default-features --lib train::: 47 passed (rerun after the review fixes, at the final HEAD).
• cargo nextest run -p zone_comfy --no-default-features --lib --stress-count 40 --test-threads 40 over the four wait_prompt timing tests: 40/40 iterations, 160 passes.
• zone-verify.sh … scoped - -- -p zone_server --test chat_context_store_tests --stress-count 5 --test-threads 10: 5/5 iterations, 125 passes, against a disposable Postgres.
• zone-verify.sh … scoped - -- -p zone_server --lib --test chat_context_store_tests --no-fail-fast --test-threads 8: 2974 passed, 40 skipped (zone_server lib tests, including the boxed Routing tests, plus the store tests).
• cargo clippy -p zone_server -p zone_comfy --all-targets --no-default-features --features zone_context/test-utils -- -D warnings: clean at the final HEAD.
• cargo fmt --all --check: clean at the final HEAD.
• The stress and zone-verify runs above came before the review fixes. Those fixes only refactor the ComfyUI code and test helpers and add an assertion message to the chat test, and the ComfyUI tests were rerun afterwards.

Not verified:

• The full ready workspace run was not repeated; the verifier runs it.
• I did not reproduce the chat lease loss under real load. Its cause comes from the code: the test never renewed its 30 s lease and took 39 s. Lease renewal itself is already covered by renewal_runs_independently_and_loss_wakes_cancellation.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 1 minute.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

• Configuration used: Repository: abnegate/zone/.coderabbit.yaml
• Review profile: CHILL
• Plan: Advanced
• Run ID: 1029c896-4084-411a-aed6-a0a1f35951b5

📥 Commits

Reviewing files that changed from the base of the PR and between a376d11 and 5935179.

📒 Files selected for processing (3)

• runner/zone_comfy/src/train.rs
• runner/zone_server/src/ws/chat.rs
• runner/zone_server/tests/chat_context_store_tests.rs

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}