From 7f8ff74d641aa22f975d88fd56b97f2a43787625 Mon Sep 17 00:00:00 2001 From: Richard McDaniel <1130888+rmcdaniel@users.noreply.github.com> Date: Fri, 13 Nov 2026 22:14:12 +0000 Subject: [PATCH 1/2] Prepare pending native fired-timer redrive guidance --- docs/failures-and-recovery.md | 2 +- docs/polyglot/server-api-reference.md | 23 +++++++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/docs/failures-and-recovery.md b/docs/failures-and-recovery.md index 4fc6805c793..07d06809675 100644 --- a/docs/failures-and-recovery.md +++ b/docs/failures-and-recovery.md @@ -264,7 +264,7 @@ Workflow releases before 2.18.0 accept a narrower set of runs. Every step before Releases before 2.19.0 ignore `reused_recorded_at` on reused signal, memo and search-attribute steps. After a Workflow 2.19.0 or later successor reuses those steps, keep 2.19.0 or later workers, or confirm that no such history can replay, before you roll back. In embedded mode, Workflow 2.19.0 replays, imports and resumes a 2.20.0 successor that reused an update or a chained signal wait the same way 2.20.0 does ([qualification receipt](https://github.com/durable-workflow/workflow/issues/864#issuecomment-6617902412)). -From 3.7.0, native Server redrive carries memos and search attributes set at start to the successor, and `describe()` reports `can_redrive` and `redrive_blocked_reason` under `actions`. From 3.8.0 it also reuses side effects and version markers before the failed step, recording each with its original payload, `reused_from_run_id` and `reused_recorded_at`, so workflow time after them matches the failed run. A 3.8.0 Server refuses to redrive a run whose workflow code upserted a memo or search attribute (`service_redrive_history_not_qualified`) or whose side effect result is stored externally (`external_side_effect_result_not_supported`). Otherwise it still applies the rules from before 2.18.0, so a fired timer is still refused with `unsupported_history`. Native Server support for the remaining rules is tracked in [server#980](https://github.com/durable-workflow/server/issues/980). +From 3.7.0, native Server redrive carries memos and search attributes set at start to the successor, and `describe()` reports `can_redrive` and `redrive_blocked_reason` under `actions`. From 3.8.0 it also reuses side effects and version markers before the failed step, recording each with its original payload, `reused_from_run_id` and `reused_recorded_at`. Native 3.8.0–3.10.x retains existing reuse timestamp strings but writes whole-second fallbacks for new reuse clocks, so those writes do not establish fractional replay-clock equality. These releases refuse workflow-code memo/search-attribute upserts (`service_redrive_history_not_qualified`), externally stored side-effect results (`external_side_effect_result_not_supported`) and fired timers (`unsupported_history`). The [pending native plain-timer and clock-precision correction](./polyglot/server-api-reference.md#native-redrive-timers) has separate publication and installed-SDK gates. Core's precision/reference follow-through and remaining native parity stay tracked in [server#980](https://github.com/durable-workflow/server/issues/980). ## Service Entry Refusal and Recovery {#service-entry-recovery} diff --git a/docs/polyglot/server-api-reference.md b/docs/polyglot/server-api-reference.md index 9ba5b802165..4159caab05f 100644 --- a/docs/polyglot/server-api-reference.md +++ b/docs/polyglot/server-api-reference.md @@ -499,6 +499,29 @@ encode them for you. A direct HTTP caller must use the official Avro encoder's `{codec, blob}` envelope or the runtime external reference below. The surrounding HTTP document is JSON; raw business JSON is not an alternative payload codec. +### Native fired-timer redrive and replay clocks {#native-redrive-timers} + +:::note Pending native publication and consumer qualification + +The plain-timer extension and fractional-clock correction are merged in [Server source 5221ed82](https://github.com/durable-workflow/server/pull/1215), with native 3.11.0 [release preparation](https://github.com/durable-workflow/server/pull/1218) still pending. The selected published Server retains its earlier timer refusal and clock-writing behavior. This source description does not establish a published producer, SDK replay result or stable selection. + +::: + +For an otherwise eligible run that failed at an activity, the new source accepts a completed plain timer before that failure. Its unique step must contain exactly one ordered `TimerScheduled` / `TimerFired` pair with matching source `timer_id`. The successor writes the pair with one fresh shared timer ID, creates no timer row or timer task, and resumes at the failed activity without waiting for that timer again. Source history stays unchanged. + +The reused `TimerFired` names the immediate predecessor in `reused_from_run_id`. Its `reused_recorded_at` retains the original timer event time across repeated redrives. Reused activities and markers also retain full available fractional precision when a fallback clock is first written. The initial replay anchor comes from the persisted source run start and is carried in `replayed_started_at`; consumed steps use their event-time authority. Repeated redrives preserve already-recorded clock strings and the existing initial anchor. Fractions omitted by an older producer cannot be reconstructed. Deadline formatting is unchanged. + +| Refusal | Timer boundary retained by the new source | +| --- | --- | +| `unsupported_timer` | A scope, parallel-group, cancellation-cleanup, condition or signal-timeout timer is outside the plain-timer slice. | +| `incomplete_timer_step` | The pair is missing, reversed, duplicated, or has absent/non-string/mismatched source timer IDs. | + +Cancelled timers remain `unsupported_history`. Step collisions, gaps and invalid failure boundaries keep their existing explicit refusals. Definition/fingerprint, parent/local-activity, external-payload and workflow-code-upsert limits still apply. Signals, updates and child workflows gain no native redrive support from this slice. Describe and redrive share structural eligibility; the existing worker-fingerprint comparison still applies when redrive is requested. + +The [actual PHP/Python/Rust cold-worker cohort](https://github.com/durable-workflow/server/issues/980#issuecomment-6865541932) must retain the same workflow definition/fingerprint through cold replacement and repeated redrive. It compares parsed full-precision UTC instants against an observed nonzero-fraction source `TimerFired`, preserves source history, and checks original callback counts, fresh timer identity and absence of repeated timer work. Whole-second rendering cannot establish that precision result, and installed history must not be rewritten to supply it. Source-controlled timestamp tests are separate evidence. + +Immutable producer/platform/source qualification, the applicable installed SDK receipts and actual Chart publication precede stable adoption. [Core's precision/reference follow-through](https://github.com/durable-workflow/workflow/issues/992) remains separate under [Server #980](https://github.com/durable-workflow/server/issues/980). The change adds no ordinary-entry clock support to an SDK that does not advertise it, and unchanged SDKs need no release solely to align version numbers. Protocol, OpenAPI and catalog shapes are unchanged. + ## Namespace And Storage Namespace routes require operator or admin roles. Mutating namespace and From 9f21e54a2e4b65ac9a9e302fafadb70f12d62c00 Mon Sep 17 00:00:00 2001 From: Richard McDaniel <1130888+rmcdaniel@users.noreply.github.com> Date: Fri, 13 Nov 2026 22:14:57 +0000 Subject: [PATCH 2/2] Clarify incomplete timer-step refusal wording --- docs/polyglot/server-api-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/polyglot/server-api-reference.md b/docs/polyglot/server-api-reference.md index 4159caab05f..963a9193f76 100644 --- a/docs/polyglot/server-api-reference.md +++ b/docs/polyglot/server-api-reference.md @@ -514,7 +514,7 @@ The reused `TimerFired` names the immediate predecessor in `reused_from_run_id`. | Refusal | Timer boundary retained by the new source | | --- | --- | | `unsupported_timer` | A scope, parallel-group, cancellation-cleanup, condition or signal-timeout timer is outside the plain-timer slice. | -| `incomplete_timer_step` | The pair is missing, reversed, duplicated, or has absent/non-string/mismatched source timer IDs. | +| `incomplete_timer_step` | A timer step has a missing event, reversed or duplicate events, or absent/non-string/mismatched source timer IDs. | Cancelled timers remain `unsupported_history`. Step collisions, gaps and invalid failure boundaries keep their existing explicit refusals. Definition/fingerprint, parent/local-activity, external-payload and workflow-code-upsert limits still apply. Signals, updates and child workflows gain no native redrive support from this slice. Describe and redrive share structural eligibility; the existing worker-fingerprint comparison still applies when redrive is requested.