[rmcdaniel]

Closes #168.

What this adds

Worker.status() returns a read-only WorkerStatus snapshot (with WorkerState and per-role WorkerRoleStatus) for an application health endpoint or an orchestrator probe. Taking a snapshot never does network I/O, claims tasks, renews leases or waits on the event loop, so a probe thread can still answer when the loop is stalled.

State	Ready	Live
not started / starting (registering)	no (not_started / registering)	no before run(), yes while registering
registered	yes only if every configured role is running, its latest Server exchange succeeded, and no request is overdue	yes while the event loop ticks, no role has exited, and no request has stalled
draining	no (draining)	yes
stopped	no	no
failed	no (failed:registration / runtime / shutdown)	no

The roles are heartbeat, workflow_poll, activity_poll and query_poll; only configured ones are reported. Two bounds apply to an outstanding request:

• request_overdue_seconds affects readiness. It defaults to one HTTP attempt deadline, so a Server outage reports server_unavailable:<role> even while transport retries continue.
• request_stall_seconds affects liveness. It defaults to twice the client's complete transport retry budget, so an outage alone never triggers a liveness restart.

Idle long polls, poll-capacity backpressure and long-running activity handlers don't affect either signal. Recovery happens on the next successful exchange, with no restart. The snapshot carries identity, states, reasons, ages and exception class names, never payloads, messages or credentials.

Also included: a stdlib probe example (examples/worker_health_probe.py, serving /readyz, /livez and /status from its own thread with SIGTERM drain), a guide (docs/guides/worker-health.md, with Kubernetes startup/readiness/liveness probes and shutdown guidance that keeps shutdown_timeout, termination grace and handler termination distinct), a reference section, a changelog entry, and the new module in the built-package smoke list.

Existing worker behavior is unchanged. The only runtime additions are in-memory progress records and a one-second local tick task that does no I/O. One reorder: the pollers classify capacity backpressure (a pure check) before the existing storage-admission re-raise.

Design correction found by connected testing

My first version used one stall bound for both signals. Against a paused native Server, the outstanding poll never failed, because transport retries keep the logical request in flight. Readiness and liveness flipped together at the stall bound, which would have restarted workers during a transient outage. That split is why the overdue (readiness) and stall (liveness) bounds now exist.

Evidence (local, macOS, Python 3.12.4)

• Unit: tests/test_worker_status.py has 24 transition tests covering not started, registering, registered, only configured roles, idle and in-flight long polls (overdue → unready but live; stalled → not live), transient ConnectError loss and recovery without re-registration, backpressure staying ready, heartbeat failure and recovery, registration refusal, poller failure, a dead query thread, an event-loop stall observed from another thread, a long-running sync activity plus drain to stopped, failed shutdown, no I/O during snapshots, JSON shape, argument validation and default bounds. They passed on 3 consecutive runs.
• Full unit suite: 3,740 passed, 4 skipped. The 2 test_public_boundary_scanner.py tests were deselected (macOS bash 3.2 lacks mapfile).
• Lint, types, docs: ruff check src tests, mypy src/durable_workflow and mkdocs build --strict all pass.
• Connected, published native Server 3.0.1 (index sha256:485295819c7739d3e937a2c89b8593532de0d9c2fbc8f2e7aaddb0b45fe4b319, disposable SQLite container): tests/integration/test_worker_status.py passes. It covers startup to ready, 5 s of idle long polls staying ready, a blocked sync activity with the query thread answering, drain, and stop.
• Connected outage (Server paused with docker pause, client timeout 10 s): ready at t=0.3 s; paused at t=0.7 s; server_unavailable:workflow_poll while still live from t=10.7 s; still unready and live at t=15.7 s; unpaused at t=15.9 s; ready again at t=18.4 s with no restart and a single registration; a new workflow then completed; stopped at t=19.2 s.

Not covered: CI's PHP-server integration job will be the first PHP run, and there's no run against PHP 2.11.8 locally.

🤖 Generated with Claude Code

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[rmcdaniel]

Exact head b7e0666e50c98717f8e2a8ab3af444006172b6ca: all 21 hosted checks pass, none failing or pending.

Changes since the first push:

• Regression corpus: the worker.py poll-loop edits triggered the replay corpus guard. Added tests/fixtures/replay_regressions/worker-status-outage-recovery-cold-history.json and a test that a worker observed unready during an outage restores readiness and completes the recorded cold history without rescheduling. The workflow is registered in the corpus replayer test.
• CI-only flake: the dead-query-thread test failed on 3.10, 3.13 and 3.15 under full-suite load. Mocked empty polls were hot-looping and starving the query thread. They now yield 2 ms, and waits allow 10 s. Locally it passed 15 of 15 runs on 3.13 in isolation.
• Sonar: extracted the run/run_until registration and role startup, the heartbeat request, and the role-liveness checks into helpers, with no behavior change. Replaced a NaN self-comparison with math.isfinite, split composite assertions, and dropped an unused example variable.

Local at this head: full unit 3,742 passed / 4 skipped (2 bash-3.2 scanner tests deselected); ruff and mypy pass. The connected native 3.0.1 integration test passes. The outage probe still holds: unready server_unavailable:workflow_poll but live from about 10 s, ready again 2.5 s after unpause, a single registration, then a clean stop.